happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge 780e76dfd9 into e9a0caed87

This commit is contained in:
copybara-service[bot] 2024-04-19 22:52:07 +05:30 committed by GitHub
parent e9a0caed87 780e76dfd9
commit 9322090834
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 17 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
DEVELOPERS.md
View File

@ -83,6 +83,23 @@ A `.clang-format` configuration is provided with our defaults, please run source
files through `clang-format` (or a formatter that produces equivalent behavior)
before finalizing PR for submission.
## Converting weights
We use a stripped down binary blob (.sbs) artifact to accelerate weight loading
in C++. These files can be downloaded directly from Kaggle and HuggingFace. You
can also convert Pytorch or Keras checkpoints to .sbs, but most end users should
not have to do this.
If starting with Keras, first run this script to convert to Pytorch:
https://github.com/keras-team/keras-nlp/blob/master/tools/gemma/export_gemma_to_torch_xla.py
From Pytorch, use the following script to generate uncompressed weights:
https://github.com/google/gemma.cpp/blob/dev/util/convert_weights.py
Then run gemma/compress_weights.cc (Bazel target :compress_weights), specifying
the resulting file as `--weights` and the desired .sbs name as the
`--compressed_weights`.
## Compile-Time Flags (Advanced)
There are several compile-time flags to be aware of (note these may or may not