f324a3b715
chat : only remove double bos/eos if added ( #15086 )
...
* only remove double bos/eos if added
* fix tests
2025-08-05 20:43:36 +02:00
be42642581
readme : update hot topics ( #15097 )
2025-08-05 20:19:33 +03:00
3306ceabf0
sycl: fix mul_mat selection ( #15092 )
2025-08-05 18:39:55 +02:00
88854c9179
Refactor legacy mode
2025-08-05 14:16:45 +01:00
c81de6e107
Fix `glm4moe` bug ( #15088 )
2025-08-05 13:56:44 +01:00
4c3fea89d6
Update report layout
2025-08-05 13:32:59 +01:00
49996a19da
Refactor variable names
2025-08-05 13:32:46 +01:00
aea9b31db5
Make ZD Score two-tailed
2025-08-05 12:57:13 +01:00
22f060c9c4
webui: fix markdown table ( #15081 )
...
* webui: fix markdown table
* webui: fix table display with themes
2025-08-05 13:56:44 +02:00
906548a00a
Update aggregated sum of squared activations per layer
2025-08-05 12:06:19 +01:00
ee3a9fcf88
context : fix index overflow on huge outputs ( #15080 )
...
* context : fix overflow when re-ordering huge outputs
* context : fix logits size overflow for huge batches
2025-08-05 11:27:45 +02:00
b37393423d
Compute aggregated (per layer) l2 norm
2025-08-05 08:54:57 +01:00
5e40cf4f1c
Do not resize if in_sum is null
2025-08-05 00:18:53 +01:00
ec428b02c3
llama : add --n-cpu-moe option ( #15077 )
...
* llama : add --n-cpu-moe option
Keeps the MoE weights of the first N layers in the CPU
2025-08-05 01:05:36 +02:00
19f68fa5a4
imatrix : warn when GGUF imatrix is saved without .gguf suffix ( #15076 )
...
* imatrix : add warning when suffix is not .gguf for GGUF imatrix
* imatrix : only warn about suffix when output format is unspecified
2025-08-04 23:26:52 +02:00
adbff66394
Merge branch 'master' into imatrix
2025-08-04 22:16:10 +01:00
c39c4e2a33
Refactor variable name
2025-08-04 22:15:50 +01:00
41613437ff
cmake: Add GGML_BACKEND_DIR option ( #15074 )
...
* cmake: Add GGML_BACKEND_DIR option
This can be used by distributions to specify where to look for backends
when ggml is built with GGML_BACKEND_DL=ON.
* Fix phrasing
2025-08-04 21:29:14 +02:00
e5bebe5251
gguf-py : add --chat-template-file to gguf_new_metadata ( #15075 )
2025-08-04 21:01:48 +02:00
ef0144c087
model: support GLM 4.5 family of models ( #14939 )
...
* model: Add GLM 4.5 (#14921 )
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Merge in PR suggestions
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* model: Add GLM 4.5 family of models (#14921 )
1. Updated tensor_mapping.py with NextN tensor mappings
- Added proper tensor mappings for all NextN/MTP tensors in /Users/samm/git/llama.cpp/gguf-py/gguf/tensor_mapping.py
- Added mappings for: eh_proj, embed_tokens, enorm, hnorm, shared_head.head, shared_head.norm
2. Added num_nextn_predict_layers configuration
- Added LLM_KV_NUM_NEXTN_PREDICT_LAYERS constant to llama-arch.h and llama-arch.cpp
- Added num_nextn_predict_layers field to llama_hparams struct
- Updated GLM4_MOE parameter loading in llama-model.cpp to read this parameter
- Modified tensor loading logic to conditionally load NextN tensors based on num_nextn_predict_layers
- Added GGUF writer support in gguf_writer.py with add_num_nextn_predict_layers() method
- Updated conversion script to extract and write this parameter from HuggingFace config
3. Added FIM tokens for GLM4_MOE
- Added GLM-4.5's FIM tokens to llama-vocab.cpp:
- <|code_prefix|> for FIM_PRE
- <|code_suffix|> for FIM_SUF
- <|code_middle|> for FIM_MID
4. Removed manual NextN tensor handling
- Removed the special-case handling in convert_hf_to_gguf.py that manually mapped NextN tensors
- NextN tensors are now handled automatically through the proper tensor mapping system
* glm 4.5 update tensors names
* model: glm 4.5 apply suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update src/llama-model.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* model: glm 4.5 apply suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* model: glm 4.5 apply suggestions from code review
* Apply suggestions from code review
* patch broken chat template
* typings fix
* add TENSOR_SKIP flag
Co-authored-by: Diego Devesa <slarengh@gmail.com>
* Update src/llama-model-loader.h
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
Co-authored-by: Diego Devesa <slarengh@gmail.com>
2025-08-04 20:29:25 +02:00
2721257e3e
quantize : fix confusing error message if ftype is invalid ( #15071 )
2025-08-04 18:11:02 +02:00
587d0118f5
ggml: WebGPU backend host improvements and style fixing ( #14978 )
...
* Add parameter buffer pool, batching of submissions, refactor command building/submission
* Add header for linux builds
* Free staged parameter buffers at once
* Format with clang-format
* Fix thread-safe implementation
* Use device implicit synchronization
* Update workflow to use custom release
* Remove testing branch workflow
2025-08-04 08:52:43 -07:00
5aa1105da2
vulkan: fix build when using glslang that does not support coopmat2 ( #15062 )
2025-08-04 07:09:19 +02:00
d31192b4ee
imatrix : use GGUF by default ( #14842 )
...
* imatrix : use GGUF by default
* imatrix : use GGUF regardless of the output filename
The legacy format can only be produced with --output-format dat
2025-08-03 22:00:05 +02:00
0a2f5496be
imatrix : fix 3d activation handling for hybrid and recurrent models ( #14994 )
...
* imatrix : use a single count for dense 3d tensors
* imatrix : fix 3d activations when model tensor is 2d
* imatrix : fix 3d tensor counts
2025-08-03 21:49:13 +02:00
11a3811164
memory : handle kv_unified for hybrid models ( #15050 )
2025-08-03 21:43:07 +02:00
97366dc6ab
vocab : JetBrains Mellum pre-tokenizer ( #15045 )
2025-08-03 21:38:18 +02:00
f1c2a4ca3f
Fix printing l2 norm when calc_mode = 1
2025-08-03 17:14:46 +01:00
90cb1be99d
Minor cosmetic changes
2025-08-03 16:57:27 +01:00
2117c4e54b
Update aggregated statistic report layout
2025-08-03 16:38:02 +01:00
a6155a8125
Add compute_layer_statistics() function
2025-08-03 16:35:03 +01:00
83bc2f288c
model : add text-only support for Kimi-VL (and find special tokens in text_config) ( #15051 )
...
* basic kimi-vl textmodel conversion
* check config["text_config"] for special tokens
2025-08-03 16:56:25 +02:00
be60469f25
Refactor function names
2025-08-03 15:10:17 +01:00
6c7a441161
vulkan: Use coopmat2 for conv2d ( #14982 )
2025-08-03 14:23:57 +02:00
fce05aac9e
Refactor lambda into compute_tensor_averages() function
2025-08-03 13:03:21 +01:00
5324558132
Update table layout
2025-08-03 10:28:47 +01:00
4d1325e1eb
Refactor variables
2025-08-03 10:28:23 +01:00
a32a2ecbed
Reformat report layout
2025-08-03 00:51:33 +01:00
4c01f51ae1
Remove inactive
2025-08-03 00:51:12 +01:00
5c0eb5ef54
opencl: fix adreno compiler detection logic ( #15029 )
2025-08-02 19:51:18 +02:00
fc8f92596f
Update table display
2025-08-02 16:46:27 +01:00
ee2509f563
Adjust threshold
2025-08-02 16:45:56 +01:00
9b841eb696
Compute l2 norm
2025-08-02 16:45:09 +01:00
b7fb362d8e
Compute cosine similarity based on activations
2025-08-02 16:43:49 +01:00
cce514a392
Compute entropy for activations
2025-08-02 16:40:40 +01:00
9744a4a1c6
Determine calculation mode
2025-08-02 16:36:12 +01:00
78ddb475de
Fix problem up when GGUF does not have in_sum
2025-08-02 16:31:21 +01:00
03d4698218
CUDA: use mma FA kernel for gqa > 4 on RTX 4000 ( #15035 )
2025-08-02 16:37:08 +02:00
3303c19b16
cuda: make im2col a little faster ( #15025 )
2025-08-02 17:15:36 +03:00
4fdea540bd
kv-cache : skip alignment of n_stream in kv-cache log msg [no ci] ( #15040 )
...
This commit removes the right alignment the `n_stream` value in the
log message in the `llama_kv_cache_unified` constructor.
The motivation for this change is to enhance the readability of log
message. Currently the output looks like this:
```console
llama_kv_cache_unified: size = 2048.00 MiB ( 4096 cells, 32 layers, 1/ 1 seqs), K (f16): 1024.00 MiB, V (f16): 1024.00 MiB
```
Notice that the `n_stream` value is right aligned, which makes it a
little harder to read.
With the change in this commit the output will look like
```console
llama_kv_cache_unified: size = 2048.00 MiB ( 4096 cells, 32 layers, 1/1 seqs), K (f16): 1024.00 MiB, V (f16): 1024.00 MiB
```
2025-08-02 17:14:57 +03:00
Sigbjørn Skjæret
Georgi Gerganov
Romain Biessy
Ed Addario
Juk Armstrong
Alex Wu
compilade
Diego Devesa
Christian Kastner
Sam
Reese Levine
Jeff Bolz
Csaba Kecskemeti
Gabriel Larson
lhez
Johannes Gäßler
leejet
Daniel Bevenius