f31e12e63b
Improved blob diff: parallel, tolerance for float
...
PiperOrigin-RevId: 724060325
2025-02-06 13:46:28 -08:00
9f5159ff68
Public visibility for compression/
...
PiperOrigin-RevId: 723529541
2025-02-05 08:53:51 -08:00
7ccc6abe87
Allow conversion, loading and inference with NUQ.
...
PiperOrigin-RevId: 723507890
2025-02-05 07:45:54 -08:00
8a6edff319
Base interleaved handling for 4.5-bit NUQ, specifically Enc, DecompressAndZeroPad, and Dec2. Includes tests.
...
PiperOrigin-RevId: 721821577
2025-01-31 10:35:32 -08:00
23dac72463
Simplified interface class and example for Gemma.cpp usage.
...
PiperOrigin-RevId: 720591037
2025-01-28 08:48:27 -08:00
7af2e70321
Add python wrappers for configs and inference.
...
Enable building compression/python/compression_test using bazel.
Add default image path for image_test and paligemma_test.
PiperOrigin-RevId: 720583438
2025-01-28 08:22:03 -08:00
bcdb0d65bd
Assorted small cleanups.
...
PiperOrigin-RevId: 720548132
2025-01-28 06:09:45 -08:00
a248f76245
Allow overriding num threads despite detecting topology
...
PiperOrigin-RevId: 720188756
2025-01-27 08:57:53 -08:00
e997468496
Apply PositionalEncodingQK always in-place.
...
PiperOrigin-RevId: 718851803
2025-01-23 07:09:30 -08:00
ce807a31a1
internal change
...
PiperOrigin-RevId: 718824952
2025-01-23 05:31:11 -08:00
a60b564b88
Infra improvements (2)
...
ops.h: move CreateInvTimescale to allow calling without depending on gemma
Pass around MatMulEnv instead of pools to avoid re-creating the env
profiler.h can now be used outside SIMD code
allocator: add StepBytes and QuantumSteps
rename worker thread with package/cluster in the name
threading: add Visit* to IndexRange
PiperOrigin-RevId: 718766704
2025-01-23 01:55:19 -08:00
f37402da57
Add parameter for base_frequency to CreateInvTimeScale().
...
Extract a few local variables to make code easier to read (hopefully).
PiperOrigin-RevId: 718749053
2025-01-23 00:56:44 -08:00
a133b3d062
Tiny fix: align template parameter order with parameter order.
...
PiperOrigin-RevId: 718411494
2025-01-22 09:13:23 -08:00
9646edc908
Internal change
...
PiperOrigin-RevId: 717916568
2025-01-21 07:53:49 -08:00
f46052b5b4
Merge pull request #473 from ufownl:bugfix/migrate_weights_target
...
PiperOrigin-RevId: 717536480
2025-01-20 08:05:38 -08:00
c4398fc72d
Infra improvements:
...
allocator: support mmap, fixed Bind, add padding
bench_matmul: Add PreventElision
BUILD: add ops_test build target
matmul.h: move ConstMat here; dynamic alloc of MatMulEnv
matmul_test: remove benchmarking
replace fprintf with HWY_WARN
threading.cc: support splitting large clusters (disabled); package_idx->pkg_idx, smaller IndexRangePartition
PiperOrigin-RevId: 717512274
2025-01-20 06:22:49 -08:00
20e5ef6d2e
Add the missing `migrate_weights` target for CMake
2025-01-17 18:56:43 +08:00
493688f6f1
Allow interactive use with new single-file weight format.
...
Add section about new weights format to README.md.
Remove model_type_required parameter.
Update error handling for flags.
PiperOrigin-RevId: 715788822
2025-01-15 07:22:33 -08:00
b93231a47d
Moved the vit config fields to their own config struct
...
PiperOrigin-RevId: 715692800
2025-01-15 01:09:49 -08:00
9d40f0117e
Added ability to load/save a complete model file, including tokenizer.
...
PiperOrigin-RevId: 707914366
2024-12-19 07:59:41 -08:00
5bc356f18f
Internal change
...
PiperOrigin-RevId: 707268913
2024-12-17 15:15:57 -08:00
73766e8ee3
Small updates to the README file.
...
PiperOrigin-RevId: 707036429
2024-12-17 04:09:55 -08:00
62c70d6715
Rename ModelTraining to PromptWrapping which is a more accurate name.
...
PiperOrigin-RevId: 705881500
2024-12-13 07:45:59 -08:00
6254f2e5ca
Removed duplicated tensor sizes from weights.h by changing the constructor used for MatPtrT
...
PiperOrigin-RevId: 705085054
2024-12-11 06:30:28 -08:00
aed17396be
Make prompt wrapping more consistent and fix duplicated tokens for multi-turn.
...
Do not echo <end_of_turn> tokens to the user.
Have verbosity=0 only show the dialog.
PiperOrigin-RevId: 705021391
2024-12-11 01:52:00 -08:00
e69bc3bc1c
Added the TensorInfo arg to the compressor so the shape and scale can be output correctly to the file in future.
...
Corrected some errors in the TensorIndex.
PiperOrigin-RevId: 705014619
2024-12-11 01:26:35 -08:00
7b77909427
Fix unhandled switch warning/error
...
PiperOrigin-RevId: 704828160
2024-12-10 13:32:53 -08:00
642fc97d51
Internal change
...
PiperOrigin-RevId: 704692923
2024-12-10 06:58:32 -08:00
d8135e836f
Merge pull request #460 from ericcurtin:common
...
PiperOrigin-RevId: 704684454
2024-12-10 06:33:37 -08:00
5bbe814a53
Tiny cleanup.
...
PiperOrigin-RevId: 704636988
2024-12-10 03:34:05 -08:00
331d2ccc02
Add support for 448px resolution to PaliGemma and PaliGemma2.
...
PiperOrigin-RevId: 704361579
2024-12-09 11:38:10 -08:00
a971088ac2
Refactor `gemma/common.cc` to improve readability and safety
...
Use `std::size` for array size calculations. Replace C-style
string manipulations with `std::string` methods. Simplify
`std::transform` usage for case conversion.
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
2024-12-09 16:36:25 +00:00
66bb435121
No public description
...
PiperOrigin-RevId: 704178245
2024-12-09 00:49:36 -08:00
9dfe2a76be
Internal change
...
PiperOrigin-RevId: 702961613
2024-12-04 20:41:47 -08:00
6a34e9c547
Print cache info and update Highway version for that
...
PiperOrigin-RevId: 702318451
2024-12-03 06:31:52 -08:00
f74d496879
Threading/infra improvements.
...
* Add Parallelize*Range helpers and partitioning helpers
* Refactor Pinning class, store original affinity (required to construct another NestedPools after pinning happened)
Compress:
* prevent Compress printing stats in tests
* zero-pad tensors
Matmul:
* add matmul_unit_test (TODO) and bench_matmul
* matmul_test: change norm to row vectors (that is what is added) and include bf16 rounding error
* Prepare for L2/L3 retrieval
PiperOrigin-RevId: 700603811
2024-11-27 01:12:00 -08:00
109a4d9f85
Add a simple benchmark for batching.
...
This is a simple Gemma benchmark with a fixed batch size of 32.
PiperOrigin-RevId: 698843573
2024-11-21 10:59:49 -08:00
3d1625d8c5
Improved consistency of compressor API, and added a universal method with a target type arg.
...
Moved configs pybind up to root level.
PiperOrigin-RevId: 698743417
2024-11-21 05:27:40 -08:00
73640d2521
Added tensor_index as a single source of truth on tensor shapes/sources and transformations
...
PiperOrigin-RevId: 697903886
2024-11-19 00:25:39 -08:00
7d685a267f
Added pybind for configs.
...
Added ability to test configs for equality.
PiperOrigin-RevId: 697572671
2024-11-18 04:03:51 -08:00
36f02ef892
Internal change.
...
PiperOrigin-RevId: 696815335
2024-11-15 02:22:32 -08:00
96513a8dc3
Added a blob_compare tool that compares two sbs files that may have the blobs in a different order
...
PiperOrigin-RevId: 696458888
2024-11-14 03:26:32 -08:00
5674c33dc5
Replace CLIF SbsWriter with pybind-based gcpp extension
...
Maintains compatibility with previous version.
PiperOrigin-RevId: 696181603
2024-11-13 10:20:02 -08:00
719699f132
Make top_k a runtime argument (instead of a model argument).
...
PiperOrigin-RevId: 696170691
2024-11-13 09:48:59 -08:00
b94295b6d9
Internal changes
...
PiperOrigin-RevId: 696155630
2024-11-13 09:01:38 -08:00
e54d9cbddd
Fix Griffin model:
...
- use HalfRope position encodings
- zero-initialize the caches for each Generate at position 0
The lack of the latter made the tests in gemma_test dependent on each other.
PiperOrigin-RevId: 694509054
2024-11-08 08:30:53 -08:00
d4050a2917
Expose BlobReader::Keys()
...
PiperOrigin-RevId: 694166186
2024-11-07 10:28:39 -08:00
868b01601f
Simpler MatMul interface, vocab types, Tristate for use_spinning
...
Add Extents2D, Range2D vocab types
Matmul uses ConstMat for inputs and RowPtr for output
Move RowVectorBatch to basics.h
Separate threading.cc
Fix topology string: report cores not LPs, and #HT
Move QStride/IsMHA into LayerConfig
ImageTokens does not require make_unique.
matmul_test: no longer require template args
PiperOrigin-RevId: 692963605
2024-11-04 07:48:29 -08:00
baaa221787
Move BF16 to basics.h for easier access, and use that typedef.
...
PiperOrigin-RevId: 691422334
2024-10-30 08:09:11 -07:00
ed4091921f
Reduce time for optimize_test and use exactly one (unpinned) thread.
...
PiperOrigin-RevId: 691013413
2024-10-29 07:37:22 -07:00
Jan Wassenberg
Phil Culliton
Daniel Keysers
Apoorv Reddy
Copybara-Service
RangerUFO
Ray Smith
The gemma.cpp Authors
Eric Curtin
Stanko Novakovic
Paul Chang