a661c52990
reverting automatically removed spaces
2025-12-04 16:12:41 +01:00
c73748ab5d
Merge branch 'sf/deepseek-ocr' into sf/deepseek-ocr-cleanup
...
# Conflicts:
# gguf-py/gguf/tensor_mapping.py
2025-12-04 15:09:32 +01:00
386ba479a2
clean up
2025-12-04 15:05:58 +01:00
7451b84105
mtmd: fix tensor names for image newlines and view separator
2025-12-04 13:26:53 +00:00
b26b507c4e
mtmd: refactor code & remove unused helper functions
2025-12-03 16:23:46 +00:00
b696c54756
mtmd: remove --dsocr-mode argument
2025-12-03 14:54:16 +00:00
43dfc0c8d6
Merge branch 'sf/deepseek-ocr' of github.com:sfallah/llama.cpp into sf/deepseek-ocr
2025-12-03 07:52:26 +00:00
e20857ba59
mtmd: simplify DeepSeek-OCR dynamic resolution preprocessing
2025-12-03 07:51:12 +00:00
c914e05405
mtmd: adapt Pillow image resizing function
2025-12-03 05:18:39 +00:00
66341666fb
Merge branch 'master' into sf/deepseek-ocr
...
# Conflicts:
# convert_hf_to_gguf.py
# tools/mtmd/clip.h
# tools/mtmd/mtmd.cpp
2025-12-02 21:02:13 +01:00
c4357dcc35
Server: Change Invalid Schema from Server Error (500) to User Error (400) ( #17572 )
...
* Make invalid schema a user error (400)
* Move invalid_argument exception handler to ex_wrapper
* Fix test
* Simplify test back to original pattern
2025-12-02 17:33:50 +01:00
5d6bd842ea
server: remove default "gpt-3.5-turbo" model name ( #17668 )
...
* server: remove default "gpt-3.5-turbo" model name
* do not reflect back model name from request
* fix test
2025-12-02 11:38:57 +01:00
fd3abe849e
server: fixing naming conflict res_error in server-models.cpp ( #17679 )
2025-12-02 11:18:39 +01:00
682e6658bb
server: explicitly set exec path when create new instance ( #17669 )
...
* Revert "rm unused fn"
This reverts commit f2dbe9c087
2025-12-02 10:25:11 +01:00
cee92af553
Add context info to server error ( #17663 )
...
* fix: Add context info to server error
* chore: update webui build output
2025-12-02 09:20:57 +01:00
ecf74a8417
mtmd: add mtmd_context_params::warmup option ( #17652 )
...
* mtmd: add mtmd_context_params::warmup option
* reuse the common_params::warmup
2025-12-01 21:32:25 +01:00
ec18edfcba
server: introduce API for serving / loading / unloading multiple models ( #17470 )
...
* server: add model management and proxy
* fix compile error
* does this fix windows?
* fix windows build
* use subprocess.h, better logging
* add test
* fix windows
* feat: Model/Router server architecture WIP
* more stable
* fix unsafe pointer
* also allow terminate loading model
* add is_active()
* refactor: Architecture improvements
* tmp apply upstream fix
* address most problems
* address thread safety issue
* address review comment
* add docs (first version)
* address review comment
* feat: Improved UX for model information, modality interactions etc
* chore: update webui build output
* refactor: Use only the message data `model` property for displaying model used info
* chore: update webui build output
* add --models-dir param
* feat: New Model Selection UX WIP
* chore: update webui build output
* feat: Add auto-mic setting
* feat: Attachments UX improvements
* implement LRU
* remove default model path
* better --models-dir
* add env for args
* address review comments
* fix compile
* refactor: Chat Form Submit component
* ad endpoint docs
* Merge remote-tracking branch 'webui/allozaur/server_model_management_v1_2' into xsn/server_model_maagement_v1_2
Co-authored-by: Aleksander <aleksander.grygier@gmail.com>
* feat: Add copy to clipboard to model name in model info dialog
* feat: Model unavailable UI state for model selector
* feat: Chat Form Actions UI logic improvements
* feat: Auto-select model from last assistant response
* chore: update webui build output
* expose args and exit_code in API
* add note
* support extra_args on loading model
* allow reusing args if auto_load
* typo docs
* oai-compat /models endpoint
* cleaner
* address review comments
* feat: Use `model` property for displaying the `repo/model-name` naming format
* refactor: Attachments data
* chore: update webui build output
* refactor: Enum imports
* feat: Improve Model Selector responsiveness
* chore: update webui build output
* refactor: Cleanup
* refactor: Cleanup
* refactor: Formatters
* chore: update webui build output
* refactor: Copy To Clipboard Icon component
* chore: update webui build output
* refactor: Cleanup
* chore: update webui build output
* refactor: UI badges
* chore: update webui build output
* refactor: Cleanup
* refactor: Cleanup
* chore: update webui build output
* add --models-allow-extra-args for security
* nits
* add stdin_file
* fix merge
* fix: Retrieve lost setting after resolving merge conflict
* refactor: DatabaseStore -> DatabaseService
* refactor: Database, Conversations & Chat services + stores architecture improvements (WIP)
* refactor: Remove redundant settings
* refactor: Multi-model business logic WIP
* chore: update webui build output
* feat: Switching models logic for ChatForm or when regenerating messges + modality detection logic
* chore: update webui build output
* fix: Add `untrack` inside chat processing info data logic to prevent infinite effect
* fix: Regenerate
* feat: Remove redundant settigns + rearrange
* fix: Audio attachments
* refactor: Icons
* chore: update webui build output
* feat: Model management and selection features WIP
* chore: update webui build output
* refactor: Improve server properties management
* refactor: Icons
* chore: update webui build output
* feat: Improve model loading/unloading status updates
* chore: update webui build output
* refactor: Improve API header management via utility functions
* remove support for extra args
* set hf_repo/docker_repo as model alias when posible
* refactor: Remove ConversationsService
* refactor: Chat requests abort handling
* refactor: Server store
* tmp webui build
* refactor: Model modality handling
* chore: update webui build output
* refactor: Processing state reactivity
* fix: UI
* refactor: Services/Stores syntax + logic improvements
Refactors components to access stores directly instead of using exported getter functions.
This change centralizes store access and logic, simplifying component code and improving maintainability by reducing the number of exported functions and promoting direct store interaction.
Removes exported getter functions from `chat.svelte.ts`, `conversations.svelte.ts`, `models.svelte.ts` and `settings.svelte.ts`.
* refactor: Architecture cleanup
* feat: Improve statistic badges
* feat: Condition available models based on modality + better model loading strategy & UX
* docs: Architecture documentation
* feat: Update logic for PDF as Image
* add TODO for http client
* refactor: Enhance model info and attachment handling
* chore: update webui build output
* refactor: Components naming
* chore: update webui build output
* refactor: Cleanup
* refactor: DRY `getAttachmentDisplayItems` function + fix UI
* chore: update webui build output
* fix: Modality detection improvement for text-based PDF attachments
* refactor: Cleanup
* docs: Add info comment
* refactor: Cleanup
* re
* refactor: Cleanup
* refactor: Cleanup
* feat: Attachment logic & UI improvements
* refactor: Constants
* feat: Improve UI sidebar background color
* chore: update webui build output
* refactor: Utils imports + move types to `app.d.ts`
* test: Fix Storybook mocks
* chore: update webui build output
* test: Update Chat Form UI tests
* refactor: Tooltip Provider from core layout
* refactor: Tests to separate location
* decouple server_models from server_routes
* test: Move demo test to tests/server
* refactor: Remove redundant method
* chore: update webui build output
* also route anthropic endpoints
* fix duplicated arg
* fix invalid ptr to shutdown_handler
* server : minor
* rm unused fn
* add ?autoload=true|false query param
* refactor: Remove redundant code
* docs: Update README documentations + architecture & data flow diagrams
* fix: Disable autoload on calling server props for the model
* chore: update webui build output
* fix ubuntu build
* fix: Model status reactivity
* fix: Modality detection for MODEL mode
* chore: update webui build output
---------
Co-authored-by: Aleksander Grygier <aleksander.grygier@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-12-01 19:41:04 +01:00
7733409734
common: improve verbosity level definitions ( #17630 )
...
* common: improve verbosity level definitions
* string_format
* update autogen docs
2025-12-01 14:38:13 +01:00
95239f92b9
mtmd: simplify SAM patch embedding
2025-12-01 07:31:24 +00:00
2ba719519d
model: LFM2-VL fixes ( #17577 )
...
* Adjust to pytorch
* Add antialiasing upscale
* Increase number of patches to 1024
* Handle default marker insertion for LFM2
* Switch to flag
* Reformat
* Cuda implementation of antialias kernel
* Change placement in ops.cpp
* consistent float literals
* Pad only for LFM2
* Address PR feedback
* Rollback default marker placement changes
* Fallback to CPU implementation for antialias implementation of upscale
2025-11-30 21:57:31 +01:00
c5f4c64fe4
mtmd : add --dsocr-mode CLI argument for DeepSeek-OCR resolution control & all native resolution modes work
2025-11-30 16:57:19 +00:00
7f8ef50cce
clip: fix nb calculation for qwen3-vl ( #17594 )
2025-11-30 15:33:55 +01:00
3c136b21a3
cli: add migration warning ( #17620 )
2025-11-30 15:32:43 +01:00
55430945ef
Merge branch 'sf/deepseek-ocr' of github.com:sfallah/llama.cpp into sf/deepseek-ocr
2025-11-30 08:55:29 +00:00
ed3b7f1056
Merge remote-tracking branch 'sfallah/master' into sf/deepseek-ocr
...
# Conflicts:
# convert_hf_to_gguf.py
# src/llama-model.cpp
# src/models/deepseek2.cpp
2025-11-30 08:29:09 +01:00
ab49f094d2
server: move server-context to its own cpp|h ( #17595 )
...
* git mv
* add server-context.h
* add server-context.h
* clean up headers
* cont : cleanup
* also expose server_response_reader (to be used by CLI)
* fix windows build
* decouple server_routes and server_http
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-11-29 22:04:44 +01:00
8c32d9d96d
server: explicitly set the function name in lambda ( #17538 )
...
As [1] explained, the real debug message will be like:
"res operator(): operator() : queue result stop"
Set the name explicitly, the message is easy for debugging:
"res operator(): recv : queue result stop"
The left "operator()" is generated by 'RES_DBG() ... __func__'
[1]: https://clang.llvm.org/extra/clang-tidy/checks/bugprone/lambda-function-name.html
Signed-off-by: Haiyue Wang <haiyuewa@163.com>
2025-11-29 18:43:29 +01:00
841a4a88df
mtmd: debug CLIP-L & first working DeepSeek-OCR model
2025-11-29 16:40:50 +00:00
0874693b44
common : fix json schema with '\' in literals ( #17307 )
...
* Fix json schema with '\' in literals
* Add "literal string with escapes" test
2025-11-29 17:06:32 +01:00
ccb2f2385e
mtmd: debug CLIP-L (vit_pre_ln)
2025-11-29 07:04:14 +00:00
a488b495f7
mtmd: SAM numerically works
2025-11-29 02:17:49 +00:00
3ce7a65c2f
server: fix: /metrics endpoint returning JSON-escaped Prometheus format ( #17386 )
...
* fix: /metrics endpoint returning JSON-escaped Prometheus format
* mod: remove string overload from ok() method
2025-11-28 19:14:00 +01:00
ddf9f94389
server : add Anthropic Messages API support ( #17570 )
...
* server : add Anthropic Messages API support
* remove -@pytest.mark.slow from tool calling/jinja tests
* server : remove unused code and slow/skip on test_anthropic_vision_base64_with_multimodal_model in test_anthropic_api.py
* server : removed redundant n field logic in anthropic_params_from_json
* server : use single error object instead of error_array in streaming response handler for /v1/chat/completions and use unordered_set instead of set in to_json_anthropic_stream()
* server : refactor Anthropic API to use OAI conversion
* make sure basic test always go first
* clean up
* clean up api key check, add test
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2025-11-28 12:57:04 +01:00
e509411cf1
server: enable jinja by default, update docs ( #17524 )
...
* server: enable jinja by default, update docs
* fix tests
2025-11-27 01:02:50 +01:00
1d594c295c
clip: (minicpmv) fix resampler kq_scale ( #17516 )
...
* debug:"solve minicpmv precision problem"
* “debug minicpmv”
* Apply suggestion from @ngxson
---------
Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com>
2025-11-26 21:44:07 +01:00
b1846f1c8e
webui: add rehype plugin to restore HTML in Markdown table cells ( #17477 )
...
* webui: add rehype plugin to restore HTML in Markdown table cells
The remark/rehype pipeline neutralizes inline HTML as literal text
(remarkLiteralHtml) so that XML/HTML snippets in LLM responses display
as-is instead of being rendered. This causes <br> and <ul> markup in
table cells to show as plain text.
This plugin traverses the HAST post-conversion, parses whitelisted HTML
patterns (<br>, <ul><li>) from text nodes, and replaces them with actual
HAST element nodes. For lists, adjacent siblings must be combined first
as the AST fragmentation breaks pattern matching.
Strict validation rejects malformed markup, keeping it as raw text.
* chore: update webui build output
2025-11-25 08:01:02 +01:00
b8372eecd9
server: split server.cpp code into server/common/task/queue ( #17362 )
...
* add server-task, server-common
* add server-queue
* rm redundant includes
* move enum stop_type to server-task
* server : headers cleanup
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-11-24 14:41:53 +01:00
81533e494e
mtmd: fix danling pointer
2025-11-24 09:02:03 +00:00
40e7e6e706
mtmd: quick fix token order
2025-11-24 08:16:32 +00:00
206f8abc3c
- dynamic resizing
...
- changes are concerning PR https://github.com/sfallah/llama.cpp/pull/4
2025-11-23 20:27:02 +01:00
0c7220db56
webui: minor settings reorganization and add disable autoscroll option ( #17452 )
...
* webui: added a dedicated 'Display' settings section that groups visualization options
* webui: added a Display setting to toggle automatic chat scrolling
* chore: update webui build output
2025-11-23 18:42:00 +01:00
6dfda99c69
Merge branch 'sf/deepseek-ocr' into sf/deepseek-ocr
2025-11-23 12:29:37 +01:00
3f71188303
mtmd: correct token order
2025-11-23 09:22:00 +00:00
4cfa15fcd7
- image encoding debugged
...
- issues fixed mainly related wrong config like n_patches etc.
- configs need to be corrected in the converter
2025-11-22 16:57:34 +01:00
ee8a1488f9
mtmd: add native resolution support
2025-11-22 15:48:13 +00:00
3fcfc3ace9
Merge pull request #3 from bluebread/sf/deepseek-ocr
...
Fixed get_rel_pos & add_rel_pos_inplace operator
2025-11-22 09:33:15 +01:00
f8f66a151b
Merge branch 'sf/deepseek-ocr' of github.com:sfallah/llama.cpp into sf/deepseek-ocr
2025-11-22 02:22:48 +00:00
effe66958e
mtmd: minor changed
2025-11-22 02:09:37 +00:00
86f111f8b7
image encoding technically works but the output can't be checked singe image decoding fails
2025-11-21 20:42:14 +01:00
7b8d735c90
mtmd: fixed the wrong scaler for get_rel_pos
2025-11-21 18:04:01 +00:00
Saba Fallah
bluebread
Chad Voegele
Xuan-Son Nguyen
senhtry
Aleksander Grygier
Tarek Dakhran
Haiyue Wang
Igor Smirnov
o7si
Fredrik Hultin
Han Qingzhe
Pascal