happyz
/
gemma.cpp
mirror of https://github.com/google/gemma.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
621 Commits 20 Branches 5 Tags 35 MiB
Commit Graph

621 Commits

Author SHA1 Message Date
Eric Ye 1408784d7b Remove unused ascii banner string 
PiperOrigin-RevId: 615504786
 2024-03-13 12:21:46 -07:00
Copybara-Service a0f316d853 Merge pull request #95 from google:conversion 
PiperOrigin-RevId: 615448039
 2024-03-13 09:37:36 -07:00
pculliton f520e5c25c
Remove WIP messages. 2024-03-13 11:36:19 -04:00
RangerUFO 83ec42954f Allow changing k parameter of `SampleTopK` as a compiler flag 2024-03-13 13:55:37 +08:00
Jan Wassenberg 5fa2eb1a86 Use bf16-rounded sqrt for scaling embeddings to match Gemma 
Thanks Daniel & Michael Han for pointing this out.
https://unsloth.ai/blog/gemma-bugs

PiperOrigin-RevId: 615250003
 2024-03-12 19:15:13 -07:00
Copybara-Service 0221956b2e Merge pull request #87 from google:refactor-tidy 
PiperOrigin-RevId: 615204427
 2024-03-12 16:10:47 -07:00
Phil Culliton b6831a2256 Fixed 7B conversion. 2024-03-12 21:12:28 +00:00
austinvhuang 72247614bb fix prefill feedback off-by-1, update fetch commit hash 2024-03-12 15:10:44 -04:00
austinvhuang 4aa8d0584e Merge branch 'dev' into refactor-tidy 2024-03-12 15:01:46 -04:00
Austin Huang a9aa63fd2e Fix bazel build attempt #2 (remove //base through automation change). 
https://github.com/google/gemma.cpp/actions/runs/8252325981/job/22571549312

PiperOrigin-RevId: 615097246
 2024-03-12 10:43:26 -07:00
Austin Huang 9345b0aed5 Fix bazel build failure: 
https://github.com/google/gemma.cpp/actions/runs/8252325981/job/22571549312

PiperOrigin-RevId: 615073081
 2024-03-12 09:41:03 -07:00
Copybara-Service ccd055e06b Merge pull request #82 from google:examples 
PiperOrigin-RevId: 615066980
 2024-03-12 09:24:24 -07:00
Jan Wassenberg 0d406061c0 Detect and print build type. Refs #88 
PiperOrigin-RevId: 614906000
 2024-03-11 21:58:10 -07:00
Copybara-Service a9be065c28 Merge pull request #77 from enum-class:rmsnorm 
PiperOrigin-RevId: 614642173
 2024-03-11 06:25:57 -07:00
enum-class 6712f07ee7 Merge branch 'dev' into rmsnorm 2024-03-11 20:13:18 +08:00
Saman 683c1b5c32
Merge branch 'google:main' into rmsnorm 2024-03-11 20:01:16 +08:00
austinvhuang 60d054e041 move arg definitions out of gemma.h to app.h 2024-03-10 23:49:25 -04:00
Copybara-Service e577198fc0 Merge pull request #78 from enum-class:rmsnorm2 
PiperOrigin-RevId: 614480854
 2024-03-10 16:14:44 -07:00
austinvhuang 415464b047 fix CMakeLists typo 2024-03-10 15:41:17 -04:00
austinvhuang 5d323c00fe fix tokenizer scope 2024-03-10 13:23:16 -04:00
austinvhuang 0fc80fad05 libgemma refactor - review changes 2024-03-10 12:55:08 -04:00
austinvhuang cc5c24c4f8 remove app.h dependency + fix bazel build 2024-03-08 18:06:43 -05:00
austinvhuang 8c7b2cf61b add README, license to hello_world 2024-03-08 17:59:54 -05:00
austinvhuang 571a5449c4 update commit hash for gemma lib 2024-03-08 17:33:33 -05:00
austinvhuang 03147effbd update loader arg names: cache -> compressed_weights, model -> weights 2024-03-08 17:32:36 -05:00
austinvhuang dfd2fdc1dd Decouple gemma constructor from loader args, update hello_world example, add convenience version of constructor (no uncompressed weights) 2024-03-08 17:26:03 -05:00
austinvhuang 42e53e2da8 [WIP] simplify hello world example, add convenience function. TODO: update git hash in CMakeLists.txt of hello world after push 2024-03-08 14:56:22 -05:00
Jan Wassenberg fce5c8c967 Avoid fadvise on older Android. Fixes #84 
PiperOrigin-RevId: 613815953
 2024-03-07 22:19:22 -08:00
austinvhuang b67e28d1a0 [WIP] remove args from GetWeights, GetCompressedWeights 2024-03-08 00:00:11 -05:00
Phil Culliton 2161908f50 Added 7B support and args parsing. Still todo: more testing of 7B conversion. 2024-03-07 22:34:14 +00:00
Austin Huang 3df06f64c2
Merge pull request #81 from ufownl/feature/separated_kvcache 
Separate KV cache from GemmaImpl
 2024-03-07 10:10:11 -05:00
RangerUFO 170a9b4690 Make `CreateKVCache` a free function rather than a method 2024-03-07 15:52:55 +08:00
RangerUFO b841612e8c Separate KV cache from GemmaImpl 2024-03-07 15:47:31 +08:00
austinvhuang 6c0388e049 [WIP] refine Runtime struct definition 2024-03-07 01:14:07 -05:00
austinvhuang 49e654258d [WIP] clean up hello_world #includes and CMakeLists.txt 2024-03-07 01:04:25 -05:00
austinvhuang e781007836 [WIP] Remove InferenceArgs from hello_world example, fix ordering of LoaderArgs validation, revert ReplGemma EOT token behavior 2024-03-06 23:21:13 -05:00
austinvhuang 7042316013 [WIP] update GemmaInterface, Gemma, and Generate input parameter specs to remove InferenceArgs. TODO: update hello_world example after git commit hash is available for fetching 2024-03-06 22:22:59 -05:00
austinvhuang 0f6a4b49d5 [WIP] quality tweaks - for constants, defer float cast and use double for intermediate computations, add `model` to EOT token 2024-03-06 15:34:11 -05:00
austinvhuang 5b9d8a9936 [WIP] dev/examples branch merge 2024-03-06 15:10:48 -05:00
austinvhuang 10f7a086aa [WIP] decouple GemmaImpl from CLI args 2024-03-06 15:06:41 -05:00
Jan Wassenberg 3cdd5e524a Fix loop iteration in GeluMulToBF16 
Also attempt to speed up builders (parallel)

PiperOrigin-RevId: 613092863
 2024-03-05 23:00:47 -08:00
enum-class 843d9b0e1f fix for-loop bounderies 2024-03-06 08:25:50 +08:00
Copybara-Service c8b9675898 Merge pull request #74 from osanseviero:patch-1 
PiperOrigin-RevId: 612937722
 2024-03-05 12:49:09 -08:00
Phil Culliton c93e1a1e4d Resolved layer ordering, reshaping, MQA->MHA, and quantization. Works only for 2B. 2024-03-05 17:54:55 +00:00
enum-class bc845515b7 fix style, add kCamelCase style for constexpr in clang-tidy 2024-03-05 20:45:30 +08:00
enum-class 5f016fb433 use hwy/simd for RMSNorm(f, bf, f) calculation 2024-03-05 17:53:52 +08:00
enum-class 507d64e3e6 use hwy/simd for SquaredL2 calculation 2024-03-05 17:37:09 +08:00
Jan Wassenberg bb9b023502 Support Bazel builds. Fixes #16 
Also fix nuq/sfp-inl: warning, cast, and disable SCALAR

PiperOrigin-RevId: 612704056
 2024-03-04 22:07:25 -08:00
Omar Sanseviero 8c857b957e
Update README.md 2024-03-04 12:58:49 +01:00
Copybara-Service cd7468199c Merge pull request #65 from enum-class:narrowing-issues 
PiperOrigin-RevId: 612279564
 2024-03-03 18:51:59 -08:00