happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Document weight conversion 

PiperOrigin-RevId: 626957718
This commit is contained in:
Jan Wassenberg 2024-04-22 01:57:49 -07:00 committed by Copybara-Service
parent e9a0caed87
commit ca971ef50f
1 changed files with 17 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
DEVELOPERS.md
View File

@ -83,6 +83,23 @@ A `.clang-format` configuration is provided with our defaults, please run source
files through `clang-format` (or a formatter that produces equivalent behavior) files through `clang-format` (or a formatter that produces equivalent behavior)
before finalizing PR for submission. before finalizing PR for submission.
## Converting weights
We use a stripped down binary blob (.sbs) artifact to accelerate weight loading
in C++. These files can be downloaded directly from Kaggle and HuggingFace. You
can also convert Pytorch or Keras checkpoints to .sbs, but most end users should
not have to do this.
If starting with Keras, first run this script to convert to Pytorch:
https://github.com/keras-team/keras-nlp/blob/master/tools/gemma/export_gemma_to_torch_xla.py
From Pytorch, use the following script to generate uncompressed weights:
https://github.com/google/gemma.cpp/blob/dev/util/convert_weights.py
Then run gemma/compress_weights.cc (Bazel target :compress_weights), specifying
the resulting file as `--weights` and the desired .sbs name as the
`--compressed_weights`.
## Compile-Time Flags (Advanced) ## Compile-Time Flags (Advanced)
There are several compile-time flags to be aware of (note these may or may not There are several compile-time flags to be aware of (note these may or may not