happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

model : replaced ggml_argsort_top_k with ggml_top_k in DeepSeek V3.2 indexer implementation since the former fails for large tensors even when using CCCL.

This commit is contained in:
Stanisław Szymczyk 2026-03-22 13:42:19 +01:00
parent 02c215991c
commit e7aa89a48c
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/models/deepseek32.cpp
View File

@ -185,7 +185,7 @@ llm_build_deepseek32::llm_build_deepseek32(const llama_model & model, const llm_
cb(indexer_score, "indexer_score", il);
uint32_t n_top_k = indexer_score->ne[0] < n_indexer_top_k ? indexer_score->ne[0] : n_indexer_top_k;
ggml_tensor * top_k = ggml_cont(ctx0, ggml_argsort_top_k(ctx0, indexer_score, n_top_k));
ggml_tensor * top_k = ggml_cont(ctx0, ggml_top_k(ctx0, indexer_score, n_top_k));
cb(top_k, "top_k", il);
// prepare new kq mask - starts filled with -INFINITY