98ef358599
clean up after rebase: revert changes from build-xcframework.sh
2025-11-23 13:41:27 -05:00
9075f546ee
clean up after rebase: remove duplicated change, revert cmake files
2025-11-23 13:37:41 -05:00
fa56838893
revert CMake files
2025-11-23 13:16:06 -05:00
d570f6e951
Formating: remove trailing whitespace
2025-11-23 13:16:03 -05:00
c8de6116b7
Revert build-xcframework.sh
2025-11-23 13:15:23 -05:00
41fa1e7e4c
addressed comments: add guards in load generic, separate wmma branch for use_mmq function
2025-11-23 13:15:22 -05:00
4237fce03e
enable MMQ fallback on RDNA4
2025-11-23 13:14:38 -05:00
1dc62a795c
PR clean up, addressed comments
2025-11-23 13:14:38 -05:00
0bf9f09ae4
add comments and code clean up
2025-11-23 13:14:35 -05:00
c770ca2812
clean up merge conflicts
2025-11-23 13:12:03 -05:00
48afe04ca3
git rebase on top of master: fixing the correctness of the mat mul operations, updating layout mappings for RDNA4
2025-11-23 13:06:01 -05:00
65a4691512
adding appropriate WMMA instructions
2025-11-23 11:54:36 -05:00
9f87b491bd
first commit naive test to enable mmq for RDNA4
2025-11-23 11:54:36 -05:00
96ac5a2329
cuda : support non-contiguous i32 to i32 copy ( #17326 )
...
* support non-contiguous i32 to i32 copy
* add tests
* rename cpy_flt to cpy_scalar and reindent params
2025-11-23 11:13:34 +01:00
bc809e9c53
vulkan: Update docker image to Ubuntu 26.04 to enable glslc features ( #17439 )
...
26.04 provides these
Signed-off-by: Eric Curtin <eric.curtin@docker.com>
2025-11-23 10:29:36 +01:00
54d83bbe85
vulkan: remove a couple unnecessary switches ( #17419 )
2025-11-23 06:29:40 +01:00
4949ac0f18
ci : switch to BoringSSL on Server workflow ( #17441 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2025-11-22 21:38:19 +01:00
3f3a4fb9c3
Revive MUL_MAT_ID to perf testing ( #17397 )
2025-11-22 10:55:43 +01:00
028f93ef98
HIP: RDNA4 tensor core support for MMF ( #17077 )
...
* mmf for rdna4
* align the padding for rdna4
* forbit mul_mat_f for rdna4
* fix as comment
* remove device kernels
* add constexpr for early return
* update based on review comment
* change based on the review comment
* pass compile error
* keep code consistency
---------
Co-authored-by: zhang hui <you@example.com>
2025-11-22 00:03:24 +01:00
8e9ddba610
opencl: refine condition for kqv mm ( #17392 )
2025-11-21 14:34:48 -08:00
23bc779a6e
model : detect GigaChat3-10-A1.8B as deepseek lite ( #17420 )
...
* Detect GigaChat3-10-A1.8B as deepseek lite
Hardcodes checking number of layers to detect if lite version of deepseek.
* Add commnent identifying deepseek lite variants
deepseek lite variants include DeepSeek-V2-Lite, GigaChat3-10B-A1.8B
2025-11-21 14:51:38 +01:00
28175f857d
cmake : add option to build and link BoringSSL ( #17205 )
...
* cmake: add option to build and link BoringSSL
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* cmake : fix typo
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* cmake : disable boringssl test and asm by default
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* cmake : skip bssl
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* cmake : disable fips
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* cmake : fix cmake --install
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
* ci : use boringssl for windows and mac
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
---------
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2025-11-21 11:46:45 +01:00
9cc4080441
ci : start using OpenSSL ( #17235 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2025-11-21 11:45:00 +01:00
f1ffbba68e
vulkan: disable async for older Intel devices ( #17369 )
...
* vulkan: disable async for older Intel devices
* update detection logic
* use name string for detection
2025-11-21 09:58:17 +01:00
2370665e56
CANN: Refactor `evaluate_and_capture_cann_graph` ( #17333 )
...
* CANN: Refactor `evaluate_and_capture_cann_graph`
**Description of the problem**
* `matched_graph` is obtained even if graph mode is disabled.
* End of graph capture and graph replay are unnecessarily placed in different `if` blocks.
**Proposed solution**
* Obtain `matched_graph` only if graph mode is enabled.
* Place end of graph capture and graph reply inside the same `if` block.
* Unify graph related comments.
* Remove trailing whitespace
2025-11-21 16:23:29 +08:00
21d31e0810
ggml-hexagon: fix swiglu failure at `test-backend-ops` ( #17344 )
...
* refactor: use hvx_vec_exp_fp32_guard_inf for overflow handling in hvx_exp_f32
* feat: add fast sigmoid function with overflow guard for fp32
* refactor: replace hvx_vec_inverse_fp32 with hvx_vec_inverse_fp32_guard_inf for improved overflow handling
* feat: enhance hvx_add_scalar_f32 with overflow handling using infinity guard
* wip
* add HVX_Vector_Alias
wip
* wip
* fix: improve handling of src1 tensor in glu_swiglu_fp32_per_thread function
* fix nc
* wip
* wip
* handle nan at inverse
* wip
* fix neg
* wip
* rename
* fix hvx_vec_inverse_fp32_guard_inf to handle infinity and NaN cases correctly
* wip
* fix hvx_vec_inverse_fp32_guard_inf to handle NaN cases correctly
* wip
* wip
* wip
* fix output sign
2025-11-20 15:45:05 -08:00
dd0f321941
readme : add Unsloth exporting to GGUF in tools ( #17411 )
2025-11-20 20:07:36 +01:00
054a45c3d3
grammar: fix regression caused by #17381 ( #17412 )
...
* grammar: fix regression caused by #17381
* more readable
2025-11-20 18:35:10 +01:00
4c91f2633f
Improved file naming & structure for UI components ( #17405 )
...
* refactor: Component iles naming & structure
* chore: update webui build output
* refactor: Dialog titles + components namig
* chore: update webui build output
* refactor: Imports
* chore: update webui build output
2025-11-20 14:07:31 +01:00
92c0b387a9
grammar : fix integer overflow ( #17381 )
...
* Fix DoS / integer overflow
* Remove optional, use INT64_MAX instead as placeholder value (it's technically -1, so it fits :)
* White space
* Actually, since it's unsigned, use UINT64_MAX
2025-11-20 14:47:04 +02:00
2286a360ff
sync : ggml
2025-11-20 14:10:44 +02:00
1d321e592b
metal : fix compile on macos 11 (whisper/3533)
2025-11-20 14:10:44 +02:00
196f5083ef
common : more accurate sampling timing ( #17382 )
...
* common : more accurate sampling timing
* eval-callback : minor fixes
* cont : add time_meas impl
* cont : fix log msg [no ci]
* cont : fix multiple definitions of time_meas
* llama-cli : exclude chat template init from time measurement
* cont : print percentage of unaccounted time
* cont : do not reset timings
2025-11-20 13:40:10 +02:00
5088b435d4
convert : fix TypeError when loading base model remotely in convert_lora_to_gguf ( #17385 )
...
* fix: TypeError when loading base model remotely in convert_lora_to_gguf
* refactor: simplify base model loading using cache_dir from HuggingFace
* Update convert_lora_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* feat: add remote_hf_model_id to trigger lazy mode in LoRA converter
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2025-11-20 12:30:12 +01:00
845f200b28
ggml : Fix transposed SOLVE_TRI result ( #17323 )
...
* Did someone transpose the SOLVE_TRI result matrix? Perhaps...
* Update ggml/src/ggml-cpu/ops.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update ggml/src/ggml-cpu/ops.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2025-11-20 12:58:21 +02:00
a7784a8b1d
DGX Spark: UMA support ( #17368 )
...
* DGX Spark: UMA support
* Updates from PR feedback
* More PR feedback cleanup
* Update ggml/src/ggml-cuda/ggml-cuda.cu
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Remove trailing whitespace
* Update ggml/src/ggml-cuda/ggml-cuda.cu
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-11-20 12:32:02 +02:00
79bb743512
ggml : remove useless and error-prone variadic macros ( #17399 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2025-11-20 11:18:27 +01:00
3ae282a06f
kleidiai: fix zero-size array declaration ( #17240 )
2025-11-20 11:45:49 +02:00
5be353ec4a
ggml-cpu:add RISC-V RVV (Zvfh) optimization for FP16 vector scaling ( #17314 )
...
* ggml-cpu:add RISC-V RVV (Zvfh) optimization for FP16 vector scaling
Signed-off-by: Wang Yang <yangwang@iscas.ac.cn>
* fix comment
* fix comment 2
---------
Signed-off-by: Wang Yang <yangwang@iscas.ac.cn>
2025-11-20 08:09:18 +02:00
7d77f07325
vulkan: implement ADD1, ARANGE, FILL, SOFTPLUS, STEP, ROUND, CEIL, FLOOR, TRUNC ( #17319 )
...
* vulkan: initialize array
* vulkan: implement ADD1
* vulkan: implement ARANGE
* vulkan: implement FILL
* vulkan: implement SOFTPLUS
* vulkan: implement STEP
* vulkan: implement ROUND
* vulkan: implement CEIL
* vulkan: implement FLOOR
* vulkan: implement TRUNC
* docs: update Vulkan ops
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2025-11-19 17:29:45 +01:00
1fa4551af0
vulkan: support larger argsort ( #17313 )
...
* vulkan: support larger argsort
This is an extension of the original bitonic sorting shader that puts the
temporary values in global memory and when more than 1024 threads are needed
it runs multiple workgroups and synchronizes through a pipelinebarrier.
To improve the memory access pattern, a copy of the float value is kept with
the index value. I've applied this same change to the original shared memory
version of the shader, which is still used when ncols <= 1024.
* Reduce the number of shader variants. Use smaller workgroups when doing a single pass, for a modest perf boost
* reduce loop overhead
* run multiple cols per invocation, to reduce barrier overhead
2025-11-19 17:25:50 +01:00
2eba631b81
vulkan: Add copy_transpose shader ( #17371 )
2025-11-19 16:50:43 +01:00
99c53d6558
webui: Add a "Continue" Action for Assistant Message ( #16971 )
...
* feat: Add "Continue" action for assistant messages
* feat: Continuation logic & prompt improvements
* chore: update webui build output
* feat: Improve logic for continuing the assistant message
* chore: update webui build output
* chore: Linting
* chore: update webui build output
* fix: Remove synthetic prompt logic, use the prefill feature by sending the conversation payload ending with assistant message
* chore: update webui build output
* feat: Enable "Continue" button based on config & non-reasoning model type
* chore: update webui build output
* chore: Update packages with `npm audit fix`
* fix: Remove redundant error
* chore: update webui build output
* chore: Update `.gitignore`
* fix: Add missing change
* feat: Add auto-resizing for Edit Assistant/User Message textareas
* chore: update webui build output
2025-11-19 14:39:50 +01:00
07b0e7a5ac
convert : use self.block_count everywhere instead of reading hparams ( #17359 )
2025-11-19 11:52:38 +01:00
fd7353d5eb
cuda: fix rope fusion for gemma3 ( #17378 )
2025-11-19 18:25:05 +08:00
6fd4f95367
Fix too relaxed check on CUDA "fast copy" (can_be_transposed) condition ( #17332 )
...
* Fix too relaxed check on CUDA "fast copy" (can_be_transposed) condition
* Argh.
* Making CISC happy ;)
* Integrate CONT tests
* Use loopy loop
* Skip new tests for (B)F16 for now.
2025-11-19 10:36:33 +01:00
980b7cd17e
vulkan: force full subgroups for flash attention to fix intel subgroup crash ( #17356 )
2025-11-19 08:46:26 +01:00
c49daff5ba
ggml-cpu: Don't pass -mpowerpc64 when -mcpu already implies it ( #17308 )
2025-11-19 14:19:00 +08:00
10e9780154
chat: fix int overflow, prevent size calculation in float/double ( #17357 )
...
* chat: fix int overflow, prevent size calculation in float/double
* Update common/chat.cpp
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-11-18 19:11:53 +01:00
a045492088
vocab : call reserve() for building plamo-2-translate suffix ( #17343 )
...
Test 'Q4_K_M' quantization on https://huggingface.co/pfnet/plamo-2-translate
The 'suffix_to_score' size is 193510, it needs 19 memory allocation with final
capacity 262144 to hold the value, if not preserve the memory.
Signed-off-by: Haiyue Wang <haiyuewa@163.com>
2025-11-18 18:58:22 +01:00
jiachengjason
jiachengjason
Sigbjørn Skjæret
Eric Curtin
Jeff Bolz
Adrien Gallouët
Masato Nakasaka
yulo
lhez
ubergarm
Raul Torres
nullname
Daniel Han
Xuan-Son Nguyen
Aleksander Grygier
Piotr Wilkin (ilintar)
Georgi Gerganov
YangLe
o7si
Scott Fudally
sudhiarm
ixgbe
Giuseppe Scrivano
Aman Gupta
Ruben Ortlam
Jeremy Rand
Haiyue Wang