Jan Wassenberg aaf51898b6 Major revamp #2 of Prefill: fix token order, parallel for multi-query ... - Allocate only the required KV caches and activation batch size - Add flags for batch sizes - Const-correct interface: Span of const int. - Also clean up the KVCache arg to a span. - Move kPrefillBatchSize into RuntimeConfig and remove related global constants. PiperOrigin-RevId: 655893197		2024-07-25 03:28:55 -07:00
..
activations.h	Prep for sharding gemma.cc: split into kv_cache, tokenizer.	2024-07-02 09:31:06 -07:00
backward-inl.h	Split up ops.h into ops/ops-inl and matmul-inl	2024-07-19 11:21:48 -07:00
backward.cc	Prep for sharding gemma.cc: split into kv_cache, tokenizer.	2024-07-02 09:31:06 -07:00
backward.h	Move raw_weights into separate header, used mainly by compress_weights.	2024-06-17 06:17:02 -07:00
backward_scalar.h	Move benchmark_helper to evals/, weights_raw to compression/.	2024-07-08 01:13:23 -07:00
backward_scalar_test.cc	Refactor configurables.	2024-07-10 21:30:58 -07:00
backward_test.cc	Split up ops.h into ops/ops-inl and matmul-inl	2024-07-19 11:21:48 -07:00
common_scalar.h	Merge pull request #212 from szabadka:adam2	2024-06-07 02:25:18 -07:00
forward-inl.h	Split up ops.h into ops/ops-inl and matmul-inl	2024-07-19 11:21:48 -07:00
forward.cc	Prep for sharding gemma.cc: split into kv_cache, tokenizer.	2024-07-02 09:31:06 -07:00
forward.h	Add config for att/final cap, skip max-subtract. Fixes #278	2024-07-01 09:45:26 -07:00
forward_scalar.h	Move benchmark_helper to evals/, weights_raw to compression/.	2024-07-08 01:13:23 -07:00
optimize_test.cc	Major revamp #2 of Prefill: fix token order, parallel for multi-query	2024-07-25 03:28:55 -07:00
optimizer.cc	Use Loader/AppArgs to construct gemma_test model, simplify AcceptFunc	2024-06-11 05:53:10 -07:00
optimizer.h	Support all weight types in a single binary.	2024-06-07 09:04:45 -07:00
prompt.h	Add missing include	2024-06-04 10:29:12 +00:00
sampler.h	Add config for att/final cap, skip max-subtract. Fixes #278	2024-07-01 09:45:26 -07:00
test_util.h	Move benchmark_helper to evals/, weights_raw to compression/.	2024-07-08 01:13:23 -07:00