happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix typo

This commit is contained in:
Ed Addario 2026-01-11 18:34:10 +00:00
parent c88f288df9
commit a297a158e3
No known key found for this signature in database
GPG Key ID: E7875815A3230993
2 changed files with 5 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
tools/imatrix/README.md
View File

@ -78,11 +78,11 @@ Versions **b5942** and newer of `llama-imatrix` store data in GGUF format by def
#### Per tensor
* **∑ E[A²]**: The sum of squares of activations (Energy) for the tensor. Tensors with high "energy" contribute most to the final output. Quantization errors here propagate strongly. These tensors usually need higher precision (e.g., Q6_K vs Q4_K).
* **L₂ Norm**: Euclidean Distance from the tensor in the previous layer. Measure of transformation magnitude; higher values indicate more significant transformation on the data.
* **Min / Max / μ / σ**: Tensor elements Min, Max, Mean, and Standard Deviation.
* **H Norm**: Shannon Entropy normalized over log₂(N). Defined as $H Norm=\frac{-\sum_{i=1}^N p_i \log_2 p_i}{log_2 N}$. Used to determine how well a prompt "exercises" the model's capabilities. Higher values indicate more uniform distribution of activations. Every neuron is firing equally; hard to prune.
* **ZD**: % of elements whose ZD-score is > 1.0 (an indicator of outliers), as described in _3.1 Layer Importance Scores_ of [Layer-Wise Quantization](https://arxiv.org/abs/2406.17415).
* **∑ E[A²]**: The sum of squares of activations (Energy) for the tensor. Tensors with high "energy" contribute most to the final output. Quantization errors here propagate strongly. These tensors usually need higher precision (e.g., Q6_K vs Q4_K).
* **L₂ Norm**: Euclidean Distance from the tensor in the previous layer. Measure of transformation magnitude; higher values indicate more significant transformation on the data.
* **CosSim**: Cosine Similarity with the tensor in the previous layer. _~1.0_, the tensor output points in the exact same direction as the previous layer's tensor (the layer is refining magnitude, not direction). _< 1.0_, the layer is rotating the vector space (changing semantic meaning).
* **PCC**: Pearson Correlation Coefficient with the tensor in the previous layer. Checks for linear correlation excluding the mean shift. Similar to CosSim but centers geometric data first. Indicates if the pattern of activation changes or just the offset.
@ -90,9 +90,9 @@ Versions **b5942** and newer of `llama-imatrix` store data in GGUF format by def
Aggregated metrics per block/layer:
* **ZD**: % of this layer's concatenated tensors' elements with |Z| > 1. Indicates general "spikiness" of the layer's activations.
* **∑ E[A²]:** Total energy of the layer's concatenated tensors. Indicates the layer's overall contribution amplitude.
* **L₂ Norm:** Euclidean Distance of the layer's concatenated tensors from the previous layer’s. Global measure of transformation magnitude.
* **ZD**: % of this layer's concatenated tensors' elements with |Z| > 1. Indicates general "spikiness" of the layer's activations.
* **CosSim**: Cosine Similarity of this layer's concatenated tensors with the previous layer.
* **PCC**: Average Pearson Correlation of the tensors in the layer.

4
tools/imatrix/imatrix.cpp
View File

@ -1395,7 +1395,7 @@ static bool show_statistics(const common_params & params) {
w_lay, "Layer", sep,
w_nam, "Tensor", sep,
"Min", "Max", "Mean", "StdDev", sep,
"H_Norm", "ZD", sep,
"H Norm", "ZD", sep,
"∑ E[A²]", "CosSim", "PCC");
LOG_INF("%s\n", std::string(154, '-').c_str());
} else {
@ -1403,7 +1403,7 @@ static bool show_statistics(const common_params & params) {
w_lay, "Layer", sep,
w_nam, "Tensor", sep,
"Min", "Max", "Mean", "StdDev", sep,
"H_Norm", "ZD", sep,
"H Norm", "ZD", sep,
"∑ E[A²]", "L2 Dist", "CosSim", "PCC");
LOG_INF("%s\n", std::string(167, '-').c_str());
}