2 Commits

Author	SHA1	Message	Date
Ting Sun	87e21bbacd	bench : make n_batch and n_ubatch configurable in Batched bench (#6500) ... * bench: make n_batch and n_ubatch configurable * bench: update doc for batched bench	2024-04-05 21:34:53 +03:00
Georgi Gerganov	8c70a5ff25	batched : add bench tool (#3545) ... * batched : add bench tool * batched : minor fix table * batched-bench : add readme + n_kv_max is now configurable * batched-bench : init warm-up batch * batched-bench : pass custom set of PP, TG and PL * batched-bench : add mmq CLI arg	2023-10-11 21:25:33 +03:00