happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
852 Commits 20 Branches 5 Tags 35 MiB
Commit Graph

20 Commits

Author SHA1 Message Date
Jan Wassenberg a084d33e41 Fix Gemma3 image: ensure A matrix is packed, preallocate 
Also ignore -2 tokens

PiperOrigin-RevId: 838869988
 2025-12-01 11:47:23 -08:00
Jan Wassenberg 3e18db17f4 Avoid hard-coding kPatchSize. Thanks @Somet2mes for reporting. Fixes #762. 
PiperOrigin-RevId: 829308896
 2025-11-07 00:32:31 -08:00
Jan Wassenberg 3ed403e287 Major cleanup of profiler zones, add Caller annotation for all pool.Run 
Pass ThreadingContext instead of Pools/Profiler individually, for access to Zones
Add GCPP_ZONE helper
Add Caller argument to pool.Run to enable new stats
Remove most direct dependencies on ThreadPool, prefer ParallelFor

PiperOrigin-RevId: 822934530
 2025-10-23 01:54:24 -07:00
Phil Culliton 503aaddd65 Add 8-bit integer quantization (I8Stream) to Gemma.cpp. 
PiperOrigin-RevId: 819787856
 2025-10-15 09:25:20 -07:00
Ray Smith ee18916abf Removed the PROFILER_ZONE from the most highly called functions to reduce the overhead. 
PiperOrigin-RevId: 819739402
 2025-10-15 07:10:04 -07:00
Jan Wassenberg 035273c184 tune pool kSpin mode in threading_context 
Previously, this happened concurrently with the matmul autotune, which could lead to incorrect outcomes.

threading: de-singleton Pinning (no longer stores affinity); pass PoolWorkerMapping; fix Pool dtor order
Also enable SPR target (Zen4 is AMD-only),
update Highway version for renamed Thread()->GlobalIdx().
PiperOrigin-RevId: 816223017
 2025-10-07 08:36:26 -07:00
Jan Wassenberg 56186193c1 Replace mt19937 with new generator to enable parallel sampling 
Split it into immutable AesCtrEngine and RngStream
Also add RowSpan and Logits span

PiperOrigin-RevId: 803336423
 2025-09-04 23:49:10 -07:00
Jan Wassenberg 229bd078a1 1.29x speedup: bf16 C1/C2. Extend most ops to any type, expand test coverage. 
Also increase dot_test.cc range for Zen4, and matmul_test tolerance (failing in some configs)

PiperOrigin-RevId: 801789922
 2025-09-01 06:34:04 -07:00
Jan Wassenberg faa4102992 (Resubmit) Prepare profiler annotations for new API 
Pass hwy::Profiler& to low-level functions.
Used ThreadingContext arg instead of NestedPools.
Use new PROFILER_ZONE3.

PiperOrigin-RevId: 794461159
 2025-08-13 01:38:24 -07:00
The gemma.cpp Authors a2d9133f7d Prepare profiler annotations for new API 
Pass hwy::Profiler& to low-level functions.
Used ThreadingContext arg instead of NestedPools.
Use new PROFILER_ZONE3.

PiperOrigin-RevId: 793865287
 2025-08-11 17:51:38 -07:00
Jan Wassenberg 4cbf63e6f0 Prepare profiler annotations for new API 
Pass hwy::Profiler& to low-level functions.
Used ThreadingContext arg instead of NestedPools.
Use new PROFILER_ZONE3.

PiperOrigin-RevId: 793821255
 2025-08-11 15:34:52 -07:00
Jan Wassenberg d1638587f0 1.14x batch decode speedup: parallelize RMSNorm ops 
Activations was over-parallelized, use single pool instead.
Also improve profiler zone annotations,
pass through worker args (for tracking concurrency), now non-optional.

PiperOrigin-RevId: 788790976
 2025-07-30 00:55:45 -07:00
Jan Wassenberg e76e29ce11 De-singleton ThreadingContext so callers can pass in their own 
weights.cc: fix BindB argument for bf16 tensors
threading_test: enable autotune
PiperOrigin-RevId: 785763618
 2025-07-22 02:08:46 -07:00
Jan Wassenberg 4bc44d5678 Minor: ModelWeightsPtrs -> WeightsPtrs 
PiperOrigin-RevId: 781954533
 2025-07-11 06:11:51 -07:00
Jan Wassenberg 0f70f285e0 1.1x prefill and decode speedup (attention/activations) 
Optimizations
- Better load-balancing in attention threading
(Previously, clusters were limited by #heads)
- Add MulByConstTo to avoid zero-init
- Parallel activations

Cleanup
- Prepare for RowPtr in A or B
- Pass through thread_id to ops
- Avoid warning in bench_matmul

PiperOrigin-RevId: 773723423
 2025-06-20 08:59:53 -07:00
Jan Wassenberg cd80d8b24d Speed up builds by skipping rarely used targets 
Centralize previous code into GEMMA_DISABLED_TARGETS

PiperOrigin-RevId: 772433723
 2025-06-17 05:44:20 -07:00
Jan Wassenberg 6773e4517c Split Activations into Griffin/Attention to reduce memory usage for attention-only tests. 
PiperOrigin-RevId: 772025282
 2025-06-16 07:52:59 -07:00
Jan Wassenberg c027a45a2e MatPtr-ify KV, shared div_seq_len, --seq_len flag 
PiperOrigin-RevId: 770194455
 2025-06-11 09:49:38 -07:00
Jan Wassenberg 6ee628ba38 Further cleanup: separate MatMulEnv arg 
move row_ptrs into MatMulEnv
Consistent arg order: layer, activations, kv_cache, env

PiperOrigin-RevId: 767886386
 2025-06-05 20:48:32 -07:00
Jan Wassenberg 3a266c662c Split gemma-inl into separate source files 
weights, mat: zero-initialize padding, required since the MatMul "avoid B decompress" optimization.

PiperOrigin-RevId: 767562313
 2025-06-05 05:36:44 -07:00