11 Commits

Author	SHA1	Message	Date
Nitin Gangahar	2d0b2c8129	Internal change. ... PiperOrigin-RevId: 822643310	2025-10-22 10:24:59 -07:00
Jan Wassenberg	4e062d68f7	Update BlobWriter comments, WriteAll->Finalize ... PiperOrigin-RevId: 790792133	2025-08-04 10:01:38 -07:00
Jan Wassenberg	799c264df3	Pre-tune thread pool before matmul ... Also improve profiler annotations - remove near-zero ones and add more for startup PiperOrigin-RevId: 789352414	2025-07-31 08:45:26 -07:00
Charles Zhao	50ee1a3e92	Write SBS progressively. ... (1) Directly write to file in BlobWriter::Add and destruct the MatOwner to release the rams. (2) Write a fake header to indicate this is V2, and write correct header and directory at the end of the file. (3) Tested on loading sbs written the old way, and new way, both worked. PiperOrigin-RevId: 789306837	2025-07-31 06:05:38 -07:00
Jan Wassenberg	2141d4788d	Add IsAppendOnly flag to file and if true, disable parallel writes ... PiperOrigin-RevId: 788805810	2025-07-30 01:51:37 -07:00
Jan Wassenberg	56c9196eb6	Add blob_path to config deduction message ... PiperOrigin-RevId: 782188689	2025-07-11 18:58:56 -07:00
Jan Wassenberg	cb188d4a0e	Fix RowT issue and improve Griffin (currently still broken) ... Use type-safe MatPtrT via dynamic_cast, avoid/remove unsafe RowT activations: Griffin tensors are now padded Griffin: add batching support, fix conv1d_cache allocation weights: bundle to TensorToRead, add kNoPad flag, fix SplitW1 const-correct fix for ForEachTensor blob_store: move BlobIO2 to .cc and rename BlobIO PiperOrigin-RevId: 760610094	2025-05-19 07:02:10 -07:00
Jan Wassenberg	c443adee33	3.8x speedup of weights loading via preadv on Linux ... Also move BlobReader reading functionality to weights.cc PiperOrigin-RevId: 759240310	2025-05-15 11:55:15 -07:00
Jan Wassenberg	a0ff98ea60	Entirely remove constexpr on PaddedDirEnd. Refs #551 ... Apparently GCC 9.4 does not handle HWY_CXX17_CONSTEXPR as we intend. PiperOrigin-RevId: 755967709	2025-05-07 12:48:19 -07:00
Jan Wassenberg	e9ecb7794d	Fix gcc build error and gemma3 crash, thanks @ufownl, fixes #551 ... PiperOrigin-RevId: 755729478	2025-05-07 00:59:18 -07:00
Jan Wassenberg	c8d92948f4	Move fields, io* and blob* from compression/ into io/ ... PiperOrigin-RevId: 755445712	2025-05-06 11:17:19 -07:00

Renamed from compression/blob_store.cc (Browse further)