happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
434 Commits 20 Branches 5 Tags 35 MiB
Commit Graph

21 Commits

Author SHA1 Message Date
Jan Wassenberg 07c34cb18a Further nuq_test speedups to prevent timeout 
PiperOrigin-RevId: 670863385
 2024-09-04 00:49:44 -07:00
Jan Wassenberg 4033ed9e78 Avoid duplication of RMSNorm, support all activation/weight types 
Add test for RMSNorm
Rename VectorizedRopeAndMulBy -> RopeAndMulBy

Move test_util to util/

PiperOrigin-RevId: 668332927
 2024-08-28 01:26:55 -07:00
Jan Wassenberg 2308514e5a Experiment with compensated dot product. 
ULP difference vs exact is 0..1, vs 200-5000 for previous.
Runtime overhead is 2.5-4x for f32 input.

PiperOrigin-RevId: 668084019
 2024-08-27 12:05:35 -07:00
Jan Wassenberg 301dc8067a Major MatMul update, 1.9-2.3x speedup on Zen4 via bf16 mul 
Supports converting all weight/activation formats to native MulT (bf16/f32)

Also:
- ConstMat/MutableMat for const correctness
- Move RowVectorBatch to allocator.h so it can be used from Matmul
- Add matmul.h so MatMulEnv can be used from Activations
- Remove kMaxThreads, detect from PerClusterPools
- Build fix: -inl.h files must be textual_hdrs, and highway.h should precede -inl.h

```
zen4 new
64, 24576, 3072, add=0, MatTA=bf16, MatTB=sfp:   616.6 GFLOPS.
64, 3072, 24576, add=0, MatTA=bf16, MatTB=sfp:   460.7 GFLOPS.
64, 24576, 3072, add=0, MatTA=f32, MatTB=sfp:    598.6 GFLOPS.
64, 3072, 24576, add=0, MatTA=f32, MatTB=sfp:    435.6 GFLOPS.

zen4 old
64, 24576, 3072, add=0, MatTA=f32, MatTB=sfp:    257.5 GFLOPS.
64, 3072, 24576, add=0, MatTA=f32, MatTB=sfp:    231.9 GFLOPS.
```

PiperOrigin-RevId: 663729812
 2024-08-16 07:52:20 -07:00
The gemma.cpp Authors 6c57feb52f Automated Code Change 
PiperOrigin-RevId: 663622838
 2024-08-16 00:01:24 -07:00
Jan Wassenberg cbb67b4ee0 Move benchmark_helper to evals/, weights_raw to compression/. 
PiperOrigin-RevId: 650155983
 2024-07-08 01:13:23 -07:00
Jan Wassenberg f823371691 Cleanup: move util/compress and convert_weights to compression/ 
Also remove unused models/, lint convert_weights

PiperOrigin-RevId: 649613088
 2024-07-05 04:16:52 -07:00
Jan Wassenberg 5c3e5f7038 Remove no longer required stats.h - use Highway version instead 
PiperOrigin-RevId: 640440379
 2024-06-05 01:37:48 -07:00
Jan Wassenberg 22fe9809ac Fix SVE build: add missing hn:: 
PiperOrigin-RevId: 632481097
 2024-05-10 06:49:26 -07:00
Jan Wassenberg e9a0caed87 Further improve IO, enable multiple backends without -D. 
Move Path into io.h and use for opening files.
Removes dependency of gemma_lib on args.
Separate Windows codepath instead of emulating POSIX functions.

Plus lint fixes.

PiperOrigin-RevId: 626279004
 2024-04-19 00:40:29 -07:00
Jan Wassenberg a8ceb75f43 Improved IO abstraction layer 
Move to unique_ptr-like File class.
Move `if OS_WIN` into wrapper functions.
exists -> Exists.

PiperOrigin-RevId: 625923056
 2024-04-17 23:15:07 -07:00
Jan Wassenberg a939b5fc9f Update distortion.h to weighted average, add distortion_test. 
More thorough checks in sfp_test and nuq_test.
nuq_test: use deterministic input generator.

PiperOrigin-RevId: 625602019
 2024-04-17 01:44:19 -07:00
Jan Wassenberg 24add61dd9 Fix SFP/NUQ for bf16 rounding in Highway 
SFP: Avoid rounding twice, and more robust TestDot.
NUQ: also more robust SNR, minor touchups to header.

PiperOrigin-RevId: 618030096
 2024-03-21 19:06:19 -07:00
Jan Wassenberg ba86c8d590 Remove obsolete copybara tags, faster bazel builds (debug) 
PiperOrigin-RevId: 617576799
 2024-03-21 04:19:02 +01:00
Eric Ye 89be4c3de8 No public description 
PiperOrigin-RevId: 617315030
 2024-03-21 04:18:36 +01:00
Jan Wassenberg 30b8a3c1ac Fix build for RPi, missing hn::. Refs #112, thanks long568 
PiperOrigin-RevId: 617704418
 2024-03-20 20:07:49 -07:00
Jan Wassenberg 06cea2bcdb Remove obsolete copybara tags, faster bazel builds (debug) 
PiperOrigin-RevId: 617576799
 2024-03-20 23:37:39 +01:00
Eric Ye ffd02c59ad No public description 
PiperOrigin-RevId: 617315030
 2024-03-20 23:37:12 +01:00
Jan Wassenberg 7d5364bb80 Remove obsolete copybara tags, faster bazel builds (debug) 
PiperOrigin-RevId: 617576799
 2024-03-20 11:31:59 -07:00
Jan Wassenberg bb9b023502 Support Bazel builds. Fixes #16 
Also fix nuq/sfp-inl: warning, cast, and disable SCALAR

PiperOrigin-RevId: 612704056
 2024-03-04 22:07:25 -08:00
Ikko Eltociear Ashimine e4e02a17d4 Copybara import of the project: 
--
5c7dbc6599 by Ikko Eltociear Ashimine <eltociear@gmail.com>:

Update build.yml

dispath -> dispatch

COPYBARA_INTEGRATE_REVIEW=https://github.com/google/gemma.cpp/pull/22 from eltociear:patch-1 5c7dbc6599
PiperOrigin-RevId: 609827161
 2024-02-23 22:32:51 -08:00