0. initialize dest with zeros
1. embd is not a vector anymore
2. embd size from embd_size may be smaller than batch_tokens because
it honors logits array, so we use ctx->n_outputs to bound our
embd outer loop.
3. Remove the batch_tokens foot-gun parameter since we have
authoritative information on the size of the embedding outputs
from the context.
4. improve comment docs
5. incorporate new usage for gritlm example
* add gritlm example
* gritlm results match
* tabs to spaces
* comment out debug printing
* rebase to new embed
* gritlm embeddings are back babeee
* add to gitignore
* allow to toggle embedding mode
* Clean-up GritLM sample code.
* Fix types.
* Flush stdout and output ending newline if streaming.
* mostly style fixes; correct KQ_mask comment
* add causal_attn flag to llama_cparams
* gritml : minor
* llama : minor
---------
Co-authored-by: Douglas Hanley <thesecretaryofwar@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Matt Grosso
Georgi Gerganov
DAN™