gemma.cpp/backprop
Jan Wassenberg 8d0882b966 Huge refactor of weight handling and model loading.
Weight handling:
- new ModelStore2 supports both pre-2025 multi-file and single-file formats
- simpler ForEachTensor with TensorArgs
- tensors are constructed with their full suffixed name

I/O:
- support mmap and stride
- Simplified SbsWriter, single insert(); add SbsReader

Misc:
- kMockTokenizer: allow creating with unavailable tokenizer
- configs.h: Simpler enum validity checks via kSentinel
- matmul.h: remove unused enable_bind (now in allocator.h)
- tensor_info: single TensorInfoRegistry class, rename from tensor_index.h

Frontends:
- Replace Allocate/CreateGemma with ctor(LoaderArgs, MatMulEnv&)
- Deduce model/weight type, remove --model and parsing
- Replace most common.h includes with configs.h
- Remove --compressed_weights, use --weights instead
- Remove ModelInfo, replaced by ModelConfig.

Backprop:
- Reduce max loss, remove backward_scalar_test (timeout)
- Update thresholds because new RandInit changes rng eval order and thus numerics
PiperOrigin-RevId: 755317484
2025-05-06 04:44:21 -07:00
..
activations.h Major refactor of allocator/args: 2025-04-10 01:29:54 -07:00
backward-inl.h Cleanup: include fixes/comments, fix leak, vector reserve 2025-04-22 12:01:46 -07:00
backward.cc Major refactor of allocator/args: 2025-04-10 01:29:54 -07:00
backward.h Major refactor of allocator/args: 2025-04-10 01:29:54 -07:00
backward_scalar.h Major refactor of allocator/args: 2025-04-10 01:29:54 -07:00
backward_test.cc Huge refactor of weight handling and model loading. 2025-05-06 04:44:21 -07:00
common_scalar.h Major refactor of allocator/args: 2025-04-10 01:29:54 -07:00
forward-inl.h Cleanup: include fixes/comments, fix leak, vector reserve 2025-04-22 12:01:46 -07:00
forward.cc Major refactor of allocator/args: 2025-04-10 01:29:54 -07:00
forward.h Eliminated TConfig. 2024-10-17 05:04:22 -07:00
forward_scalar.h Major refactor of allocator/args: 2025-04-10 01:29:54 -07:00
optimize_test.cc Huge refactor of weight handling and model loading. 2025-05-06 04:44:21 -07:00
optimizer.cc Huge refactor of weight handling and model loading. 2025-05-06 04:44:21 -07:00
optimizer.h Huge refactor of weight handling and model loading. 2025-05-06 04:44:21 -07:00
prompt.h Add missing include 2024-06-04 10:29:12 +00:00
sampler.h Add config for att/final cap, skip max-subtract. Fixes #278 2024-07-01 09:45:26 -07:00
test_util.h Huge refactor of weight handling and model loading. 2025-05-06 04:44:21 -07:00