ce111d39d6
opencl: add fused `rms_norm_mul` ( #14841 )
...
* opencl: add fused `rms_norm` + `mul`
* opencl: improve workgroup size for `rms_norm_mul`
2025-07-25 17:12:13 +02:00
e7fecba934
docs : update HOWTO‑add‑model.md for ModelBase and new model classes ( #14874 )
...
This patch updates the example in docs/development/HOWTO-add-model.md to
reflect recent changes after `TextModel` and `MmprojModel` were introduced.
It replaces the outdated `Model` base class with `TextModel` or `MmprojModel`
and updates the registration example accordingly.
Signed-off-by: Wook Song <wook16.song@samsung.com>
2025-07-25 16:25:05 +02:00
e2b7621e7c
ggml : remove invalid portPos specifiers from dot files ( #14838 )
...
Neither "g" nor "x" are valid portPos specifiers per the official
[graphviz documents](https://graphviz.org/docs/attr-types/portPos/ ):
> If a compass point is used, it must have the form "n","ne","e","se","s","sw","w","nw","c","_".
I tested locally for it to fall back to default portPos specifier if an
invalid portPos is specified. As a consequence, we can remove associated
code.
2025-07-25 14:29:57 +03:00
c1dbea752a
context : restore preemptive sched reset when LLAMA_SET_ROWS=0 ( #14870 )
...
ggml-ci
2025-07-25 14:28:06 +03:00
749e0d27f0
mtmd : fix 32-bit narrowing issue in export-lora and mtmd clip ( #14503 )
...
* [fix] Fix 32-bit narrowing issue in export-lora and mtmd clip
* Update export-lora.cpp
* Update clip.cpp
* Update export-lora.cpp
* format: use space to replace tab
2025-07-25 13:08:04 +02:00
64bf1c3744
rpc : check for null buffers in get/set/copy tensor endpoints ( #14868 )
2025-07-25 12:17:02 +02:00
c12bbde372
sched : fix multiple evaluations of the same graph with pipeline parallelism ( #14855 )
...
ggml-ci
2025-07-25 11:07:26 +03:00
3f4fc97f1d
musa: upgrade musa sdk to rc4.2.0 ( #14498 )
...
* musa: apply mublas API changes
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
* musa: update musa version to 4.2.0
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
* musa: restore MUSA graph settings in CMakeLists.txt
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
* musa: disable mudnnMemcpyAsync by default
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
* musa: switch back to non-mudnn images
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
* minor changes
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
* musa: restore rc in docker image tag
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
---------
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
2025-07-24 20:05:37 +01:00
2df255da3c
sync : ggml
...
ggml-ci
2025-07-24 20:27:23 +03:00
60f816a79d
cmake : fix usage issues (ggml/1257)
...
* CMake config: Create target only once
Fix error on repeated find_package(ggml).
For simplicity, check only for the top-level ggml::ggml.
* CMake config: Add CUDA link libs
* CMake config: Add OpenCL link libs
* CMake config: Use canonical find_dependency
Use set and append to control link lib variables.
Apply more $<LINK_ONLY...>.
* CMake config: Wire OpenMP dependency
2025-07-24 20:27:23 +03:00
5592f278b6
ggml-cpu : remove stdlib include from repack.cpp (ggml/1276)
...
This commit removes the inclusion of `<cstdlib>`.
The motivation for this change is that this source file does not seem to
use any functions from this header and the comment about `qsort` is a
little misleading/confusing.
2025-07-24 20:27:23 +03:00
e4868d16d2
context : perform output reorder lazily upon access after sync ( #14853 )
...
* context : perform output reorder after lazily upon access after sync
ggml-ci
* cont : add TODO
2025-07-24 16:31:48 +03:00
820de57d4f
chat : fix kimi-k2 chat template ( #14852 )
2025-07-24 13:59:56 +02:00
f263f5d9ae
ggml-zdnn: fix missing data transform call
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 18:30:10 +08:00
1c75ed63e5
ggml-zdnn: fix compiler error missing type
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 18:22:34 +08:00
a1d8568c14
ggml-zdnn: impl matmul
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 18:13:07 +08:00
cb4a63aad6
sycl: fixed semantics of block offset calculation ( #14814 )
2025-07-24 11:09:57 +01:00
86f5623d90
llama : fix MiniCPM inference after Granite Four changes ( #14850 )
...
MiniCPM models use the llm_build_granite constructor which was changed
in the Granite Four PR to use hparams.rope_finetuned instead of a
use_rope parameter. MiniCPM models need rope enabled by default.
Fixes inference from gibberish to correct responses.
2025-07-24 11:50:51 +02:00
39cffdf188
docs: add libcurl-dev install hint for Linux distros ( #14801 )
...
* docs: add libcurl-dev install hint for Linux distros
Signed-off-by: PouyaGhahramanian <PooyaGhahramanian@gmail.com>
* Update docs/build.md
---------
Signed-off-by: PouyaGhahramanian <PooyaGhahramanian@gmail.com>
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
2025-07-24 11:26:44 +02:00
59e9805ab0
ggml-zdnn: code clean up
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 16:26:29 +08:00
c1653ab639
ggml-zdnn: fix incorrect ztensor shape, reduce memory padding
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 16:22:06 +08:00
828519659b
ggml-zdnn: update supports_op matmul matrix
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 16:11:37 +08:00
065908cb09
metal : fix fusion across different encoders ( #14849 )
...
* metal : fix fusion across different encoders
ggml-ci
* cont : add assertion
ggml-ci
2025-07-24 10:24:05 +03:00
4ec6291a24
sycl: fix undefined variable in work group size check ( #14843 )
2025-07-24 12:50:41 +08:00
18658b8607
ggml-zdnn: impl init_tensor
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 12:02:20 +08:00
a12363bbf0
convert : text-only support for GLM-4.1V-9B-Thinking ( #14823 )
...
* use language_model part only, ignore visual layers
* fix rope_dim calculation
2025-07-23 23:23:57 +02:00
a86f52b285
CUDA: fix overflow in FA, tune performance ( #14840 )
2025-07-23 21:43:25 +02:00
da2e0e70ba
ggml-zdnn: switch buffers back and set to arbitrary number
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 02:31:22 +08:00
63fbc45ed6
ggml-zdnn: switch to std vector instead of array
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 01:09:01 +08:00
b7f4b6fde3
ggml-zdnn: rework init_tensor to create new buffers
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 01:03:53 +08:00
ee0ed78d54
ggml-zdnn: add check for view tensors to prevent init_tensor
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 00:56:32 +08:00
13c64448bd
ggml-zdnn: assign tensor->extra to buffer
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 00:48:32 +08:00
13c05872f2
ggml-zdnn: implement at least 1 op to test
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 00:44:05 +08:00
9e84742e72
ggml-zdnn: test ztensor finding in init_tensor
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 00:40:22 +08:00
af9f4f0039
ggml-zdnn: fix compiler warnings and bugfixes
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 00:25:41 +08:00
b284197df4
CUDA: fix compilation with GGML_CUDA_F16 ( #14837 )
2025-07-23 18:22:30 +02:00
ae2f656d7e
ggml-zdnn: bugfix new impl
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 00:18:53 +08:00
7c6395f826
ggml-zdnn: rewrite the backend implementation
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-24 00:14:45 +08:00
221c0e0c58
ci : correct label refactor->refactoring ( #14832 )
2025-07-23 14:27:54 +02:00
04ddb2ac95
ggml-zdnn: update op out_prod to use tensor->extra
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-23 19:51:37 +08:00
77a753297b
ggml-zdnn: support op out_prod
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-23 19:28:51 +08:00
07a19e27a2
CUDA: fix quantized KV cache + multiple sequences ( #14822 )
...
* CUDA: fix quantized KV cache + multiple sequences
* Update ggml/src/ggml-cuda/fattn-common.cuh
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-07-23 14:08:09 +03:00
18f3b5ff9e
tests : add non-cont K,V FA tests
...
ggml-ci
2025-07-23 14:08:09 +03:00
7233358d29
memory : handle saving/loading null layers in recurrent memory ( #14675 )
...
* Update llama-memory-recurrent.cpp
handle saving/loading null layers in recurrent memory
* fixed styling issues and updated comments
* fix styling issue
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2025-07-23 11:16:41 +03:00
11d58d29de
ggml-zdnn: add comments to prevent accidentally deleting lines
...
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2025-07-23 14:54:44 +08:00
6c88b3bb25
ggml: fix loongarch quantize_row_q8_1 error ( #14827 )
2025-07-23 09:39:51 +03:00
14c28dfc50
CANN: weight format to NZ for Ascend310P3 ( #14407 )
...
* weight format to nz for 310p
* remove quant weight format to nz
* clean code
* fix
* make the conditions for converting weights to NZ format consistent
* clean code
2025-07-23 11:58:00 +08:00
8c988fa41d
CUDA: add fused rms norm ( #14800 )
2025-07-23 09:25:42 +08:00
acd6cb1c41
ggml : model card yaml tab->2xspace ( #14819 )
2025-07-22 19:29:43 +03:00
84712b6043
vulkan: fix rms_norm_mul to handle broadcasting dim0 ( #14817 )
2025-07-22 17:35:21 +02:00
lhez
wooksong
Oliver Simons
Georgi Gerganov
kiwi
Chris Rohlf
Diego Devesa
R0CKSTAR
Kai Pastor
Daniel Bevenius
Xuan-Son Nguyen
Aaron Teo
Alberto Cabrera Pérez
yummy
Pouya
Donghyeon Jeong
jacekpoplawski
Johannes Gäßler
Sigbjørn Skjæret
l3utterfly
lixing-star
chen fan
Aman Gupta
Csaba Kecskemeti
Jeff Bolz