llama.cpp/tools
Ben Racicot c1b911654a
server: fix router mode deadlock on child crash and TOCTOU race in models_max (#20763)
Two bugs in `server_models::load()` that affect router mode reliability:

**Bug 1: Deadlock when child process crashes**

When a child process is killed (e.g., SIGKILL from OS code signature
validation), the monitoring thread deadlocks on `stopping_thread.join()`
because the stopping_thread's wait predicate (`is_stopping`) is never
satisfied — the model name was never inserted into `stopping_models`.
`update_status()` is never reached and the model stays stuck in LOADING
state permanently.

Fix: extend the stopping_thread's wait predicate to also wake when the
child process is no longer alive (`!subprocess_alive()`). When woken by
a dead child, the thread skips the shutdown sequence and returns
immediately. The original `stopping_models.erase()` logic is preserved
for normal unloads.

**Bug 2: TOCTOU race bypasses `--models-max` (ref #20137)**

`unload_lru()` is called outside the mutex, then `load()` acquires the
lock afterward. Under concurrent requests, multiple threads observe
capacity and all proceed to load, exceeding the limit.

Fix: re-check capacity under the lock after `unload_lru()` returns.
If another thread filled the slot in the window between `unload_lru()`
and the lock acquisition, reject with an error instead of silently
exceeding the limit.
2026-03-19 22:16:05 +01:00
..
batched-bench Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
cli common/parser: add proper reasoning tag prefill reading (#20424) 2026-03-19 16:58:21 +01:00
completion common/parser: add `--skip-chat-parsing` to force a pure content parser. (#20289) 2026-03-17 16:16:43 +01:00
cvector-generator chore : correct typos [no ci] (#20041) 2026-03-05 08:50:21 +01:00
export-lora Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
fit-params llama-fit-params: keep explicit --ctx-size 0 (#19070) 2026-01-24 22:13:08 +01:00
gguf-split Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
imatrix chore : correct typos [no ci] (#20041) 2026-03-05 08:50:21 +01:00
llama-bench llama-bench: introduce `-hf` and `-hff` flags & use `--mmap 1` by default (#20211) 2026-03-09 09:05:44 +08:00
mtmd mtmd: add clip_graph::build_mm() (#20751) 2026-03-19 13:11:39 +01:00
parser common/parser: add proper reasoning tag prefill reading (#20424) 2026-03-19 16:58:21 +01:00
perplexity tools : enable kvu in perplexity for hellaswag, winogrande, multiple-choice (#19954) 2026-03-13 21:25:57 +01:00
quantize llama-quant : fail early on missing imatrix, refactor type selection, code cleanup (#19770) 2026-03-10 08:16:05 +02:00
results llama: end-to-end tests (#19802) 2026-03-08 12:30:21 +01:00
rpc Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
server server: fix router mode deadlock on child crash and TOCTOU race in models_max (#20763) 2026-03-19 22:16:05 +01:00
tokenize Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
tts Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
CMakeLists.txt llama: end-to-end tests (#19802) 2026-03-08 12:30:21 +01:00