62cef26ac5
This commit adds two targets to the Makefile for quantizing of Quantization Aware Trained (QAT) models to Q4_0 format. The motivation for this is that this sets the token embedding and the output tensors data types to Q8_0 instead of the default Q6_K. This is someting that we wish to enforce for QAT Q4_0 models that are to be uploaded to ggml-org on Huggingface to guarantee the best quality. |
||
|---|---|---|
| .. | ||
| batched | ||
| batched.swift | ||
| convert-llama2c-to-ggml | ||
| deprecation-warning | ||
| diffusion | ||
| embedding | ||
| eval-callback | ||
| gen-docs | ||
| gguf | ||
| gguf-hash | ||
| gritlm | ||
| jeopardy | ||
| llama.android | ||
| llama.swiftui | ||
| lookahead | ||
| lookup | ||
| model-conversion | ||
| parallel | ||
| passkey | ||
| retrieval | ||
| save-load-state | ||
| simple | ||
| simple-chat | ||
| simple-cmake-pkg | ||
| speculative | ||
| speculative-simple | ||
| sycl | ||
| training | ||
| CMakeLists.txt | ||
| Miku.sh | ||
| chat-13B.bat | ||
| chat-13B.sh | ||
| chat-persistent.sh | ||
| chat-vicuna.sh | ||
| chat.sh | ||
| convert_legacy_llama.py | ||
| json_schema_pydantic_example.py | ||
| json_schema_to_grammar.py | ||
| llama.vim | ||
| llm.vim | ||
| pydantic_models_to_grammar.py | ||
| pydantic_models_to_grammar_examples.py | ||
| reason-act.sh | ||
| regex_to_grammar.py | ||
| server-llama2-13B.sh | ||
| server_embd.py | ||
| ts-type-to-grammar.sh | ||