llama.cpp/tools
European Tech 6d7dc316f2 server: auto-save/restore slot state on child exit/start in router mode
In router mode with --models-max 1, switching models kills the child
process, destroying all in-memory state including the prompt cache and
context checkpoints.  This forces a full prompt re-processing on every
model swap return, which can take tens of seconds for long prompts.

This patch adds two methods (auto_save_slots, auto_restore_slots) that
are called automatically during the child process lifecycle:

- auto_save_slots: called after start_loop() returns (before clean_up),
  saves each slot's state + checkpoints to --slot-save-path using the
  model filename stem as the save name.

- auto_restore_slots: called after load_model() (before start_loop),
  checks if a save file exists for this model and restores it.

Combined with the checkpoint persistence from the previous commit,
this makes model hot-swapping fully transparent: the conversation
context is preserved across swaps with no client-side changes.

Tested with Qwen3.5-27B + Qwen3.5-35B-A3B MoE in router mode:
- Swap 27B→MoE: ~7s (incl auto-save 826 MiB state + 749 MiB checkpoints)
- Swap MoE→27B: ~6s (incl auto-restore)
- cache_n after restore: 26549 (91ms vs 23s without)
2026-03-20 22:35:58 +01:00
..
batched-bench Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
cli common/parser: add proper reasoning tag prefill reading (#20424) 2026-03-19 16:58:21 +01:00
completion common/parser: add `--skip-chat-parsing` to force a pure content parser. (#20289) 2026-03-17 16:16:43 +01:00
cvector-generator chore : correct typos [no ci] (#20041) 2026-03-05 08:50:21 +01:00
export-lora Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
fit-params llama-fit-params: keep explicit --ctx-size 0 (#19070) 2026-01-24 22:13:08 +01:00
gguf-split Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
imatrix chore : correct typos [no ci] (#20041) 2026-03-05 08:50:21 +01:00
llama-bench llama-bench: introduce `-hf` and `-hff` flags & use `--mmap 1` by default (#20211) 2026-03-09 09:05:44 +08:00
mtmd mtmd: add clip_graph::build_mm() (#20751) 2026-03-19 13:11:39 +01:00
parser common/parser: add proper reasoning tag prefill reading (#20424) 2026-03-19 16:58:21 +01:00
perplexity tools : enable kvu in perplexity for hellaswag, winogrande, multiple-choice (#19954) 2026-03-13 21:25:57 +01:00
quantize llama-quant : fail early on missing imatrix, refactor type selection, code cleanup (#19770) 2026-03-10 08:16:05 +02:00
results llama: end-to-end tests (#19802) 2026-03-08 12:30:21 +01:00
rpc Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
server server: auto-save/restore slot state on child exit/start in router mode 2026-03-20 22:35:58 +01:00
tokenize Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
tts Fix locale-dependent float printing in GGUF metadata (#17331) 2026-03-04 09:30:40 +01:00
CMakeLists.txt llama: end-to-end tests (#19802) 2026-03-08 12:30:21 +01:00