512b2c8fe4
merge with changes from https://github.com/ggml-org/llama.cpp/pull/18042
2025-12-16 14:07:04 +01:00
51c3de6887
Merge remote-tracking branch 'sfallah/master' into sf/deepseek-ocr
...
# Conflicts:
# gguf-py/gguf/constants.py
# gguf-py/gguf/tensor_mapping.py
# tools/mtmd/clip-impl.h
# tools/mtmd/clip.cpp
# tools/mtmd/models/models.h
2025-12-16 12:16:25 +01:00
3d86c6c2b5
model: support GLM4V vision encoder ( #18042 )
...
* convert ok
* no deepstack
* less new tensors
* cgraph ok
* add mrope for text model
* faster patch merger
* add GGML_ROPE_TYPE_MRNORM
* add support for metal
* move glm4v do dedicated graph
* convert: add norm_embd
* clip: add debugging fn
* working correctly
* fix style
* use bicubic
* fix mrope metal
* improve cpu
* convert to neox ordering on conversion
* revert backend changes
* force stop if using old weight
* support moe variant
* fix conversion
* fix convert (2)
* Update tools/mtmd/clip-graph.h
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* process mrope_section on TextModel base class
* resolve conflict merge
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-12-16 11:25:26 +01:00
4a4f82968c
Merge branch 'ggml-org:master' into sf/deepseek-ocr
2025-12-16 09:09:52 +01:00
96a181a933
mtmd: refactor audio preprocessing ( #17978 )
...
* mtmd: refactor audio preprocessing
* refactor
Co-authored-by: Tarek <tdakhran@users.noreply.github.com>
* wip
* wip (2)
* improve constructor
* fix use_natural_log
* fix padding for short input
* clean up
* remove need_chunking
---------
Co-authored-by: Tarek <tdakhran@users.noreply.github.com>
2025-12-15 14:16:52 +01:00
b3bf8cba05
Merge remote-tracking branch 'sfallah/master' into sf/deepseek-ocr
...
# Conflicts:
# convert_hf_to_gguf.py
2025-12-15 10:19:50 +01:00
745fa0e78b
model : add glm-asr support ( #17901 )
...
* [model] add glm-asr support
* fix format for ci
* fix convert format for ci
* update glm_asr convert script & use build_ffn for glm_asr clip & use build_stack for padding and review
* check root architecture for convert hf script
* fix conficlt with upstream
* fix convert script for glm asr & format clip-impl
* format
* restore hparams text
* improved conversion
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2025-12-15 03:18:46 +01:00
37f5a1093b
mtmd: enhance image resizing in llava_uhd ( #18014 )
2025-12-14 15:57:52 +01:00
6c36c03815
minor formatting fixes
2025-12-14 15:14:32 +01:00
f95a6fe9f3
quick and (potential) dirty merge with https://github.com/ggml-org/llama.cpp/pull/17909
2025-12-13 13:52:46 +01:00
e0e69fd3fb
Merge remote-tracking branch 'sfallah/master' into sf/deepseek-ocr-merge_#17965
...
# Conflicts:
# src/llama-kv-cache.cpp
# tools/mtmd/clip.cpp
2025-12-13 10:59:46 +01:00
e39a2ce66d
clip: move model cgraphs into their own files ( #17965 )
...
* clip: move model cgraphs into their own files
* more explicit enums
* fix linux build
* fix naming
* missing headers
* nits: add comments for contributors
2025-12-12 21:14:48 +01:00
d70f171fac
merge with changes from https://github.com/ggml-org/llama.cpp/pull/17909
...
added new opt to tests.sh to disable flash-attn
2025-12-11 10:11:27 +01:00
33fabf0bd8
Merge branch 'master' into sf/deepseek-ocr-merge-test
...
# Conflicts:
# tools/mtmd/clip.cpp
# tools/mtmd/mtmd-cli.cpp
2025-12-11 08:13:50 +01:00
c6b2c9310c
mtmd: some small clean up ( #17909 )
...
* clip: add support for fused qkv in build_vit
* use bulid_ffn whenever possible
* fix internvl
* mtmd-cli: move image to beginning
* test script: support custom args
2025-12-10 22:20:06 +01:00
ed944cd25b
fix: test-1.jpg ORC issue with small (640) resolution
...
setting min-resolution base (1024) max large (1280) for dynamic-resolution
2025-12-10 20:20:55 +01:00
4dff236a52
ggml : remove GGML_KQ_MASK_PAD constant ( #17910 )
...
* ggml : remove GGML_KQ_MASK_PAD constant
* cont : remove comment
2025-12-10 20:53:16 +02:00
5174a1e69a
mtmd: minor fix
2025-12-08 04:54:19 +00:00
48c6cf2132
mtmd: convert model in FP16
2025-12-08 02:36:00 +00:00
53273f83f8
mtmd: fixed wrong input setting
2025-12-07 23:58:22 +00:00
5dfcc5abb1
mtmd: add detailed comments for resize_bicubic_pillow
2025-12-07 10:15:09 +00:00
2d918b3e21
mtmd: make sam hparams configurable
2025-12-06 06:55:53 +00:00
15f2ada0ed
mtmd: simplify get_rel_pos
2025-12-06 06:32:41 +00:00
d981f19e9d
minor editorconfig-check fixes
2025-12-05 13:18:15 +01:00
5f2ee1aecf
Merge branch 'ggml-org:master' into sf/deepseek-ocr
2025-12-05 11:56:06 +01:00
f5bd310a5e
minor formatting and style
2025-12-05 09:30:58 +01:00
076138a428
corrected code-branch when flash-attn disabled
...
enabling usage of --flash-attn option
2025-12-04 23:45:59 +01:00
5381b9cf63
using common build_attn in sam
2025-12-04 23:13:29 +01:00
fc3f625fef
mtmd: support combined QKV projection in buid_vit
2025-12-04 17:57:43 +00:00
a661c52990
reverting automatically removed spaces
2025-12-04 16:12:41 +01:00
c73748ab5d
Merge branch 'sf/deepseek-ocr' into sf/deepseek-ocr-cleanup
...
# Conflicts:
# gguf-py/gguf/tensor_mapping.py
2025-12-04 15:09:32 +01:00
386ba479a2
clean up
2025-12-04 15:05:58 +01:00
7451b84105
mtmd: fix tensor names for image newlines and view separator
2025-12-04 13:26:53 +00:00
b26b507c4e
mtmd: refactor code & remove unused helper functions
2025-12-03 16:23:46 +00:00
b696c54756
mtmd: remove --dsocr-mode argument
2025-12-03 14:54:16 +00:00
43dfc0c8d6
Merge branch 'sf/deepseek-ocr' of github.com:sfallah/llama.cpp into sf/deepseek-ocr
2025-12-03 07:52:26 +00:00
e20857ba59
mtmd: simplify DeepSeek-OCR dynamic resolution preprocessing
2025-12-03 07:51:12 +00:00
c914e05405
mtmd: adapt Pillow image resizing function
2025-12-03 05:18:39 +00:00
a96283adc4
mtmd: fix --no-warmup ( #17695 )
2025-12-02 22:48:08 +01:00
66341666fb
Merge branch 'master' into sf/deepseek-ocr
...
# Conflicts:
# convert_hf_to_gguf.py
# tools/mtmd/clip.h
# tools/mtmd/mtmd.cpp
2025-12-02 21:02:13 +01:00
ecf74a8417
mtmd: add mtmd_context_params::warmup option ( #17652 )
...
* mtmd: add mtmd_context_params::warmup option
* reuse the common_params::warmup
2025-12-01 21:32:25 +01:00
95239f92b9
mtmd: simplify SAM patch embedding
2025-12-01 07:31:24 +00:00
2ba719519d
model: LFM2-VL fixes ( #17577 )
...
* Adjust to pytorch
* Add antialiasing upscale
* Increase number of patches to 1024
* Handle default marker insertion for LFM2
* Switch to flag
* Reformat
* Cuda implementation of antialias kernel
* Change placement in ops.cpp
* consistent float literals
* Pad only for LFM2
* Address PR feedback
* Rollback default marker placement changes
* Fallback to CPU implementation for antialias implementation of upscale
2025-11-30 21:57:31 +01:00
c5f4c64fe4
mtmd : add --dsocr-mode CLI argument for DeepSeek-OCR resolution control & all native resolution modes work
2025-11-30 16:57:19 +00:00
7f8ef50cce
clip: fix nb calculation for qwen3-vl ( #17594 )
2025-11-30 15:33:55 +01:00
55430945ef
Merge branch 'sf/deepseek-ocr' of github.com:sfallah/llama.cpp into sf/deepseek-ocr
2025-11-30 08:55:29 +00:00
ed3b7f1056
Merge remote-tracking branch 'sfallah/master' into sf/deepseek-ocr
...
# Conflicts:
# convert_hf_to_gguf.py
# src/llama-model.cpp
# src/models/deepseek2.cpp
2025-11-30 08:29:09 +01:00
841a4a88df
mtmd: debug CLIP-L & first working DeepSeek-OCR model
2025-11-29 16:40:50 +00:00
ccb2f2385e
mtmd: debug CLIP-L (vit_pre_ln)
2025-11-29 07:04:14 +00:00
a488b495f7
mtmd: SAM numerically works
2025-11-29 02:17:49 +00:00
Saba Fallah
Xuan-Son Nguyen
piDack
Haowei Wu
Georgi Gerganov
bluebread
Tarek Dakhran