happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
512 Commits 20 Branches 5 Tags 35 MiB
Commit Graph

2 Commits

Author SHA1 Message Date
Jan Wassenberg 02ce1e344f Use NestedPools, add NUMA infra 
Improved threading.h, fix thread counts for single package/cluster systems
Temporarily forces to a single socket. Prefill 29.28 tps, decode 6.92.

Also fix benchmarks.cc build, update tensor allocator to Allocator

PiperOrigin-RevId: 687307167
 2024-10-18 08:11:18 -07:00
Jan Wassenberg 2c28b18eb0 Add NestedPools: one per socket/cluster 
Use in dot_test
app.h: add new flags and rename num_threads to max_threads
matmul: Parallelize MatMulSlow and enable spinning, more large/fewer medium test cases
PiperOrigin-RevId: 683216386
 2024-10-07 09:40:19 -07:00