This commit extracts the replay_last_token function from
save-load-state.cpp to common.h.
The motivation for this is to allow reuse of the function but also to
clarify the intent of code that replays the last token after loading
the session state.
This commit updates the session handing in the completion tool to handle
the that logits are no longer stored in the session file. Instead, we
need to replay the last token to get the logits for sampling.
* completion : simplify batch (embd) processing
This commit simplifies the processing of embd by removing the for loop
that currently exists which uses params.n_batch as its increment. This
commit also removes the clamping of n_eval as the size of embd is always
at most the size of params.n_batch.
The motivation is to clarify the code as it is currently a little
confusing when looking at this for loop in isolation and thinking that
it can process multiple batches.
* add an assert to verify n_eval is not greater than n_batch
Daniel Bevenius
Georgi Gerganov
o7si
Xuan-Son Nguyen