happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
gemma.cpp/gemma
History
Jan Wassenberg ec02726cf7 6x large-batch, short-prompt prefill speedup 
Parallelize over queries instead of tokens
introduce non_eos so we only iterate over not yet EOS queries; remove TokenStreamer.
move RMSNormInplaceBatched out of Transformer to call the latter from prefill
Consistent arg order.

Fix gemma_test EOS handling which (caught by msan), remove from tokenizer.h
Also add output to gemma_batch_bench, fix name

PiperOrigin-RevId: 769676106
 		2025-06-10 09:56:20 -07:00
..
bindings Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
evals Add MMLU eval to github 2024-05-20 10:20:53 -07:00
activations.h 6x large-batch, short-prompt prefill speedup 2025-06-10 09:56:20 -07:00
attention.cc Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
attention.h Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
configs.cc Fix paligemma_test, refs #588 2025-06-03 04:45:22 -07:00
configs.h Fix paligemma_test, refs #588 2025-06-03 04:45:22 -07:00
configs_test.cc Minor: rename compression/shared -> types.h 2025-05-13 06:53:21 -07:00
gemma-inl.h Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
gemma.cc 6x large-batch, short-prompt prefill speedup 2025-06-10 09:56:20 -07:00
gemma.h Restructure internal initialization. 2025-06-10 01:25:31 -07:00
gemma_args.h 1.07x batch decode speedup: more BF16 weights and activations 2025-06-03 23:30:18 -07:00
griffin.cc Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
griffin.h Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
kv_cache.cc Fix RowT issue and improve Griffin (currently still broken) 2025-05-19 07:02:10 -07:00
kv_cache.h Fix RowT issue and improve Griffin (currently still broken) 2025-05-19 07:02:10 -07:00
model_store.cc Fix paligemma_test, refs #588 2025-06-03 04:45:22 -07:00
model_store.h Remove backprop/ 2025-05-28 07:01:17 -07:00
run.cc Restructure internal initialization. 2025-06-10 01:25:31 -07:00
tensor_info.cc Major refactor to de-templatize gemma-inl and weights 2025-06-02 23:01:35 -07:00
tensor_info.h Minor: rename compression/shared -> types.h 2025-05-13 06:53:21 -07:00
tensor_info_test.cc Major refactor to de-templatize gemma-inl and weights 2025-06-02 23:01:35 -07:00
tokenizer.cc Huge refactor of weight handling and model loading. 2025-05-06 04:44:21 -07:00
tokenizer.h 6x large-batch, short-prompt prefill speedup 2025-06-10 09:56:20 -07:00
vit.cc Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
vit.h Further cleanup: separate MatMulEnv arg 2025-06-05 20:48:32 -07:00
weights.cc Avoid warning about inability to map, unless explicitly requested 2025-06-05 09:10:08 -07:00
weights.h Split gemma-inl into separate source files 2025-06-05 05:36:44 -07:00