From ca971ef50f8dcb1c4bceb88354b67c5f7bdb1a68 Mon Sep 17 00:00:00 2001 From: Jan Wassenberg Date: Mon, 22 Apr 2024 01:57:49 -0700 Subject: [PATCH] Document weight conversion PiperOrigin-RevId: 626957718 --- DEVELOPERS.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/DEVELOPERS.md b/DEVELOPERS.md index 08565ad..865ac26 100644 --- a/DEVELOPERS.md +++ b/DEVELOPERS.md @@ -83,6 +83,23 @@ A `.clang-format` configuration is provided with our defaults, please run source files through `clang-format` (or a formatter that produces equivalent behavior) before finalizing PR for submission. +## Converting weights + +We use a stripped down binary blob (.sbs) artifact to accelerate weight loading +in C++. These files can be downloaded directly from Kaggle and HuggingFace. You +can also convert Pytorch or Keras checkpoints to .sbs, but most end users should +not have to do this. + +If starting with Keras, first run this script to convert to Pytorch: +https://github.com/keras-team/keras-nlp/blob/master/tools/gemma/export_gemma_to_torch_xla.py + +From Pytorch, use the following script to generate uncompressed weights: +https://github.com/google/gemma.cpp/blob/dev/util/convert_weights.py + +Then run gemma/compress_weights.cc (Bazel target :compress_weights), specifying +the resulting file as `--weights` and the desired .sbs name as the +`--compressed_weights`. + ## Compile-Time Flags (Advanced) There are several compile-time flags to be aware of (note these may or may not