happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
462 Commits 18 Branches 5 Tags 35 MiB
Commit Graph

5 Commits

Author SHA1 Message Date
Daniel Keysers a8e08778d4 Add an additional QueryModel() overload to GemmaEnv. 
Use args only in GemmaEnv constructor, store everything else in RuntimeConfig.
Add runtime option to turn off thread spinning.

PiperOrigin-RevId: 670467320
 2024-09-03 02:25:19 -07:00
Jan Wassenberg aaf51898b6 Major revamp #2 of Prefill: fix token order, parallel for multi-query 
- Allocate only the required KV caches and activation batch size
- Add flags for batch sizes
- Const-correct interface: Span of const int.
- Also clean up the KVCache arg to a span.
- Move kPrefillBatchSize into RuntimeConfig and remove related global constants.

PiperOrigin-RevId: 655893197
 2024-07-25 03:28:55 -07:00
Jan Wassenberg cbb67b4ee0 Move benchmark_helper to evals/, weights_raw to compression/. 
PiperOrigin-RevId: 650155983
 2024-07-08 01:13:23 -07:00
Jan Wassenberg 85fcd3cd80 Cleanup: add ModelInfo struct, remove gcpp:: 
PiperOrigin-RevId: 648707763
 2024-07-02 07:11:15 -07:00
Jan Wassenberg af8eb2fde3 Declutter gemma/ directory, move binaries to evals/ and util/. 
PiperOrigin-RevId: 648400795
 2024-07-01 09:51:04 -07:00
Renamed from gemma/benchmark.cc (Browse further)