happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
gemma.cpp/backprop
History
Jan Wassenberg b831fa8482 1.3x prefill, 0.95x decode: matmul replacing last matvec 
Before 38.28, 9.17 (with profiler enabled, prompt = 330 tok)
```
Gen.FFW                                 :      15414 x         4692352 = 24.166318
Gen.Attention.SumHeads                  :      15414 x         1394804 =  7.183451 !!
Gen.Embedding                           :        361 x        49961894 =  6.026297
Gen.Attention.QKV                       :      15414 x         1005125 =  5.176546
Gen.Attention.DotSoftmax                :      15414 x          885480 =  4.560357
RopeAndMulBy                            :     696528 x           11867 =  2.761818
```

After 49.80, 8.68
```
Gen.FFW                                 :      14448 x         5312783 = 25.646868
Gen.Embedding                           :        338 x        63044815 =  7.119845
Gen.Attention.QKV                       :      14448 x         1115003 =  5.382557
Gen.Attention.DotSoftmax                :      14448 x          897577 =  4.332957
RopeAndMulBy                            :     673344 x           11886 =  2.674156
Gen.Attention.SumHeads                  :      14448 x          518291 =  2.501993 !!
```
PiperOrigin-RevId: 662024085
 		2024-08-12 03:36:01 -07:00
..
activations.h Prep for sharding gemma.cc: split into kv_cache, tokenizer. 2024-07-02 09:31:06 -07:00
backward-inl.h 1.03-1.08x decode speedup: precompute Rope theta, fuse 2024-08-09 01:23:24 -07:00
backward.cc 1.03-1.08x decode speedup: precompute Rope theta, fuse 2024-08-09 01:23:24 -07:00
backward.h 1.03-1.08x decode speedup: precompute Rope theta, fuse 2024-08-09 01:23:24 -07:00
backward_scalar.h Move benchmark_helper to evals/, weights_raw to compression/. 2024-07-08 01:13:23 -07:00
backward_scalar_test.cc Refactor configurables. 2024-07-10 21:30:58 -07:00
backward_test.cc 1.03-1.08x decode speedup: precompute Rope theta, fuse 2024-08-09 01:23:24 -07:00
common_scalar.h Merge pull request #212 from szabadka:adam2 2024-06-07 02:25:18 -07:00
forward-inl.h 1.03-1.08x decode speedup: precompute Rope theta, fuse 2024-08-09 01:23:24 -07:00
forward.cc 1.03-1.08x decode speedup: precompute Rope theta, fuse 2024-08-09 01:23:24 -07:00
forward.h 1.03-1.08x decode speedup: precompute Rope theta, fuse 2024-08-09 01:23:24 -07:00
forward_scalar.h Move benchmark_helper to evals/, weights_raw to compression/. 2024-07-08 01:13:23 -07:00
optimize_test.cc 1.3x prefill, 0.95x decode: matmul replacing last matvec 2024-08-12 03:36:01 -07:00
optimizer.cc Use Loader/AppArgs to construct gemma_test model, simplify AcceptFunc 2024-06-11 05:53:10 -07:00
optimizer.h Support all weight types in a single binary. 2024-06-07 09:04:45 -07:00
prompt.h Add missing include 2024-06-04 10:29:12 +00:00
sampler.h Add config for att/final cap, skip max-subtract. Fixes #278 2024-07-01 09:45:26 -07:00
test_util.h Move benchmark_helper to evals/, weights_raw to compression/. 2024-07-08 01:13:23 -07:00