happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

server : enable backend sampling for multiple outputs per sequence

This commit is contained in:
Daniel Bevenius 2026-02-25 15:39:35 +01:00
parent 765998f2d7
commit 2235b4be49
No known key found for this signature in database
1 changed files with 0 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
tools/server/server-context.cpp
View File

@ -1149,9 +1149,6 @@ private:
backend_sampling &= task.params.sampling.backend_sampling; backend_sampling &= task.params.sampling.backend_sampling;
// TODO: speculative decoding requires multiple samples per batch - not supported yet
backend_sampling &= !(slot.spec && task.params.speculative.n_max > 0);
// TODO: getting post/pre sampling logits is not yet supported with backend sampling // TODO: getting post/pre sampling logits is not yet supported with backend sampling
backend_sampling &= !need_logits; backend_sampling &= !need_logits;