happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix fix for weight type define, refs #198 

GEMMA_WEIGHT_T is indeed the correct flag for the C++ compiler,
but the readme references CMake, and there the correct flag name is WEIGHT_TYPE.

PiperOrigin-RevId: 641170380
This commit is contained in:
Jan Wassenberg 2024-06-07 01:31:43 -07:00 committed by Copybara-Service
parent 8dc0e5ea83
commit e3f4374e81
1 changed files with 7 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@ -151,13 +151,15 @@ cmake -B build
**or** if you downloaded bfloat16 weights (any model *without* `-sfp` in the **or** if you downloaded bfloat16 weights (any model *without* `-sfp` in the
name), instead of running cmake with no options as above, run cmake with name), instead of running cmake with no options as above, run cmake with
GEMMA_WEIGHT_T set to [highway's](https://github.com/google/highway) WEIGHT_TYPE set to [highway's](https://github.com/google/highway)
`hwy::bfloat16_t` type. We intend to soon support all weight types without `hwy::bfloat16_t` type. Alternatively, you can also add
requiring compiler flags. Note that we recommend using `-sfp` weights instead of `-DGEMMA_WEIGHT_T=hwy::bfloat16_t` to the C++ compiler flags.
bfloat16 for faster inference.
We intend to soon support all weight types without requiring extra flags. Note
that we recommend using `-sfp` weights instead of bfloat16 for faster inference.
```sh ```sh
cmake -B build -DGEMMA_WEIGHT_T=hwy::bfloat16_t cmake -B build -DWEIGHT_TYPE=hwy::bfloat16_t
``` ```
After running whichever of the above `cmake` invocations that is appropriate for After running whichever of the above `cmake` invocations that is appropriate for