52 Commits

Author	SHA1	Message	Date
Georgi Gerganov	921772104b	speculative : add grammar support (#2991) ... * speculative : add grammar support * grammars : add json_arr.gbnf * grammar : add comments to new grammar file * grammar : remove one nested level * common : warm-up with 2 tokens - seems to work better * speculative : print draft token pieces * speculative : reuse grammar parser + better logs and comments * speculative : avoid grammar_mem * make : fix speculative build	2023-09-05 08:46:17 +03:00
Georgi Gerganov	47068e5170	speculative : PoC for speeding-up inference via speculative sampling (#2926) ... * speculative : initial example * speculative : print encoding speed * speculative : add --draft CLI arg	2023-09-03 15:12:08 +03:00