c26c3402fe
chore: update webui build output
2025-11-21 11:10:07 +01:00
049f40dfdf
refactor: Use only the message data `model` property for displaying model used info
2025-11-21 11:00:49 +01:00
f1ffbba68e
vulkan: disable async for older Intel devices ( #17369 )
...
* vulkan: disable async for older Intel devices
* update detection logic
* use name string for detection
2025-11-21 09:58:17 +01:00
45bf2a4983
Merge remote-tracking branch 'ngxson/xsn/server_model_management_v1_2' into allozaur/server_model_management_v1_2
2025-11-21 09:25:17 +01:00
2370665e56
CANN: Refactor `evaluate_and_capture_cann_graph` ( #17333 )
...
* CANN: Refactor `evaluate_and_capture_cann_graph`
**Description of the problem**
* `matched_graph` is obtained even if graph mode is disabled.
* End of graph capture and graph replay are unnecessarily placed in different `if` blocks.
**Proposed solution**
* Obtain `matched_graph` only if graph mode is enabled.
* Place end of graph capture and graph reply inside the same `if` block.
* Unify graph related comments.
* Remove trailing whitespace
2025-11-21 16:23:29 +08:00
21d31e0810
ggml-hexagon: fix swiglu failure at `test-backend-ops` ( #17344 )
...
* refactor: use hvx_vec_exp_fp32_guard_inf for overflow handling in hvx_exp_f32
* feat: add fast sigmoid function with overflow guard for fp32
* refactor: replace hvx_vec_inverse_fp32 with hvx_vec_inverse_fp32_guard_inf for improved overflow handling
* feat: enhance hvx_add_scalar_f32 with overflow handling using infinity guard
* wip
* add HVX_Vector_Alias
wip
* wip
* fix: improve handling of src1 tensor in glu_swiglu_fp32_per_thread function
* fix nc
* wip
* wip
* handle nan at inverse
* wip
* fix neg
* wip
* rename
* fix hvx_vec_inverse_fp32_guard_inf to handle infinity and NaN cases correctly
* wip
* fix hvx_vec_inverse_fp32_guard_inf to handle NaN cases correctly
* wip
* wip
* wip
* fix output sign
2025-11-20 15:45:05 -08:00
cc88f6a75b
chore: update webui build output
2025-11-21 00:08:09 +01:00
4bf82a10f1
feat: Improved UX for model information, modality interactions etc
2025-11-21 00:05:43 +01:00
a2e912cf35
address review comment
2025-11-20 21:54:22 +01:00
cd5c699304
add docs (first version)
2025-11-20 21:45:05 +01:00
be25bccdff
address review comment
2025-11-20 21:37:22 +01:00
dd0f321941
readme : add Unsloth exporting to GGUF in tools ( #17411 )
2025-11-20 20:07:36 +01:00
6929c9f43d
address thread safety issue
2025-11-20 18:38:02 +01:00
054a45c3d3
grammar: fix regression caused by #17381 ( #17412 )
...
* grammar: fix regression caused by #17381
* more readable
2025-11-20 18:35:10 +01:00
5369aaa1d6
address most problems
2025-11-20 18:34:22 +01:00
216140867e
tmp apply upstream fix
2025-11-20 18:19:21 +01:00
c35dee3bd7
Merge remote-tracking branch 'ngxson/xsn/server_model_management_v1_2' into allozaur/server_model_management_v1_2
2025-11-20 16:36:45 +01:00
8a88576849
refactor: Architecture improvements
2025-11-20 16:34:25 +01:00
5805ca7960
add is_active()
2025-11-20 16:26:31 +01:00
d0ea9e0830
also allow terminate loading model
2025-11-20 16:20:14 +01:00
6610724f8e
fix unsafe pointer
2025-11-20 16:13:30 +01:00
b9ebdf616a
more stable
2025-11-20 15:49:40 +01:00
55d33a8b8c
feat: Model/Router server architecture WIP
2025-11-20 14:24:50 +01:00
919d3f8cbf
Merge branch 'master' into xsn/server_model_management_v1_2
2025-11-20 14:19:16 +01:00
4c91f2633f
Improved file naming & structure for UI components ( #17405 )
...
* refactor: Component iles naming & structure
* chore: update webui build output
* refactor: Dialog titles + components namig
* chore: update webui build output
* refactor: Imports
* chore: update webui build output
2025-11-20 14:07:31 +01:00
92c0b387a9
grammar : fix integer overflow ( #17381 )
...
* Fix DoS / integer overflow
* Remove optional, use INT64_MAX instead as placeholder value (it's technically -1, so it fits :)
* White space
* Actually, since it's unsigned, use UINT64_MAX
2025-11-20 14:47:04 +02:00
7c6eb17fad
fix windows
2025-11-20 13:14:56 +01:00
2286a360ff
sync : ggml
2025-11-20 14:10:44 +02:00
1d321e592b
metal : fix compile on macos 11 (whisper/3533)
2025-11-20 14:10:44 +02:00
196f5083ef
common : more accurate sampling timing ( #17382 )
...
* common : more accurate sampling timing
* eval-callback : minor fixes
* cont : add time_meas impl
* cont : fix log msg [no ci]
* cont : fix multiple definitions of time_meas
* llama-cli : exclude chat template init from time measurement
* cont : print percentage of unaccounted time
* cont : do not reset timings
2025-11-20 13:40:10 +02:00
5088b435d4
convert : fix TypeError when loading base model remotely in convert_lora_to_gguf ( #17385 )
...
* fix: TypeError when loading base model remotely in convert_lora_to_gguf
* refactor: simplify base model loading using cache_dir from HuggingFace
* Update convert_lora_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* feat: add remote_hf_model_id to trigger lazy mode in LoRA converter
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2025-11-20 12:30:12 +01:00
845f200b28
ggml : Fix transposed SOLVE_TRI result ( #17323 )
...
* Did someone transpose the SOLVE_TRI result matrix? Perhaps...
* Update ggml/src/ggml-cpu/ops.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* Update ggml/src/ggml-cpu/ops.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2025-11-20 12:58:21 +02:00
a7784a8b1d
DGX Spark: UMA support ( #17368 )
...
* DGX Spark: UMA support
* Updates from PR feedback
* More PR feedback cleanup
* Update ggml/src/ggml-cuda/ggml-cuda.cu
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* Remove trailing whitespace
* Update ggml/src/ggml-cuda/ggml-cuda.cu
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-11-20 12:32:02 +02:00
79bb743512
ggml : remove useless and error-prone variadic macros ( #17399 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2025-11-20 11:18:27 +01:00
3ae282a06f
kleidiai: fix zero-size array declaration ( #17240 )
2025-11-20 11:45:49 +02:00
5be353ec4a
ggml-cpu:add RISC-V RVV (Zvfh) optimization for FP16 vector scaling ( #17314 )
...
* ggml-cpu:add RISC-V RVV (Zvfh) optimization for FP16 vector scaling
Signed-off-by: Wang Yang <yangwang@iscas.ac.cn>
* fix comment
* fix comment 2
---------
Signed-off-by: Wang Yang <yangwang@iscas.ac.cn>
2025-11-20 08:09:18 +02:00
0ef3b61e82
add test
2025-11-20 00:29:59 +01:00
5423d42a35
use subprocess.h, better logging
2025-11-20 00:05:29 +01:00
54b3545791
fix windows build
2025-11-19 22:30:47 +01:00
abc0ca478a
does this fix windows?
2025-11-19 22:24:00 +01:00
399f536dc7
fix compile error
2025-11-19 21:33:44 +01:00
fc5901a449
server: add model management and proxy
2025-11-19 21:23:00 +01:00
7d77f07325
vulkan: implement ADD1, ARANGE, FILL, SOFTPLUS, STEP, ROUND, CEIL, FLOOR, TRUNC ( #17319 )
...
* vulkan: initialize array
* vulkan: implement ADD1
* vulkan: implement ARANGE
* vulkan: implement FILL
* vulkan: implement SOFTPLUS
* vulkan: implement STEP
* vulkan: implement ROUND
* vulkan: implement CEIL
* vulkan: implement FLOOR
* vulkan: implement TRUNC
* docs: update Vulkan ops
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2025-11-19 17:29:45 +01:00
1fa4551af0
vulkan: support larger argsort ( #17313 )
...
* vulkan: support larger argsort
This is an extension of the original bitonic sorting shader that puts the
temporary values in global memory and when more than 1024 threads are needed
it runs multiple workgroups and synchronizes through a pipelinebarrier.
To improve the memory access pattern, a copy of the float value is kept with
the index value. I've applied this same change to the original shared memory
version of the shader, which is still used when ncols <= 1024.
* Reduce the number of shader variants. Use smaller workgroups when doing a single pass, for a modest perf boost
* reduce loop overhead
* run multiple cols per invocation, to reduce barrier overhead
2025-11-19 17:25:50 +01:00
2eba631b81
vulkan: Add copy_transpose shader ( #17371 )
2025-11-19 16:50:43 +01:00
99c53d6558
webui: Add a "Continue" Action for Assistant Message ( #16971 )
...
* feat: Add "Continue" action for assistant messages
* feat: Continuation logic & prompt improvements
* chore: update webui build output
* feat: Improve logic for continuing the assistant message
* chore: update webui build output
* chore: Linting
* chore: update webui build output
* fix: Remove synthetic prompt logic, use the prefill feature by sending the conversation payload ending with assistant message
* chore: update webui build output
* feat: Enable "Continue" button based on config & non-reasoning model type
* chore: update webui build output
* chore: Update packages with `npm audit fix`
* fix: Remove redundant error
* chore: update webui build output
* chore: Update `.gitignore`
* fix: Add missing change
* feat: Add auto-resizing for Edit Assistant/User Message textareas
* chore: update webui build output
2025-11-19 14:39:50 +01:00
07b0e7a5ac
convert : use self.block_count everywhere instead of reading hparams ( #17359 )
2025-11-19 11:52:38 +01:00
fd7353d5eb
cuda: fix rope fusion for gemma3 ( #17378 )
2025-11-19 18:25:05 +08:00
6fd4f95367
Fix too relaxed check on CUDA "fast copy" (can_be_transposed) condition ( #17332 )
...
* Fix too relaxed check on CUDA "fast copy" (can_be_transposed) condition
* Argh.
* Making CISC happy ;)
* Integrate CONT tests
* Use loopy loop
* Skip new tests for (B)F16 for now.
2025-11-19 10:36:33 +01:00
980b7cd17e
vulkan: force full subgroups for flash attention to fix intel subgroup crash ( #17356 )
2025-11-19 08:46:26 +01:00
Aleksander Grygier
Jeff Bolz
Raul Torres
nullname
Xuan Son Nguyen
Daniel Han
Piotr Wilkin (ilintar)
Georgi Gerganov
YangLe
o7si
Scott Fudally
Adrien Gallouët
sudhiarm
ixgbe
Giuseppe Scrivano
Sigbjørn Skjæret
Aman Gupta
Ruben Ortlam