happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,089 Commits 559 Branches 5,903 Tags 1.4 GiB
Commit Graph

13 Commits

Author SHA1 Message Date
Georgi Gerganov d2b10302ce
improve grader 2026-02-15 22:12:02 +02:00
Georgi Gerganov 68dde884d6
minor 2026-02-15 21:21:40 +02:00
Georgi Gerganov fd90796da2
eval : support multiple dataset runs 2026-02-15 21:08:24 +02:00
Georgi Gerganov 8156d549f6
sim : fix answer matching 2026-02-15 21:08:24 +02:00
Georgi Gerganov fb1481d60d
eval : add prompts 2026-02-15 21:08:24 +02:00
Georgi Gerganov 812ae13ec1
eval : print progress 2026-02-15 21:08:24 +02:00
Georgi Gerganov e79e8d02d5
examples: add task summary table to llama-eval-new.py 2026-02-15 21:08:23 +02:00
Georgi Gerganov 62b04cef54
examples: add threading support and model parameter to llama-eval-new.py 
- Add ThreadPoolExecutor for parallel request processing controlled by --threads
- Add --model argument to specify model name in request data
- Refactor process() to use thread-safe _process_single_case() method
- Update progress tracking to work with concurrent execution
 2026-02-15 21:08:23 +02:00
Georgi Gerganov c2619c18bf
examples: use cached dataset path to avoid HF Hub requests 2026-02-15 21:08:23 +02:00
Georgi Gerganov 87f8930968
examples: remove HF_HUB_OFFLINE to allow dataset download 2026-02-15 21:08:23 +02:00
Georgi Gerganov 9453f9de12
examples: use HF_HUB_OFFLINE to avoid HF Hub warnings 2026-02-15 21:08:23 +02:00
Georgi Gerganov 5a1be6ce37
examples: implement flexible grader system for answer validation 
- Add Grader class supporting regex and CLI-based grading
- Implement built-in regex patterns for AIME, GSM8K, MMLU, HellaSwag, ARC, WinoGrande
- Add CLI grader interface: python script.py --answer <pred> --expected <gold>
- Add HF telemetry disable to avoid warnings
- Support exact match requirement for regex patterns
- Add 30-second timeout for CLI grader
- Handle both boxed and plain text formats for AIME answers
 2026-02-15 21:08:23 +02:00
Georgi Gerganov 5cc2258e82
examples: add simplified llama-eval-new.py for AIME evaluation 
- Create new simplified evaluation script focused only on AIME
- Implement EvalState and Processor dataclasses for structured state management
- Add real-time feedback showing correct/incorrect status per case
- Abstract grading interface for external grader support
- Use structured JSON output for eval state
- Apply HuggingFace dataset caching to avoid repeated downloads
- Remove Levenshtein matching - eval script only sends requests and validates answers
 2026-02-15 21:08:22 +02:00