663a0b8cce
Quant models run with accuracy issue
2026-01-15 11:19:15 -08:00
d4ca760da8
Add quant weight conversion functions from genai gguf reader
2026-01-15 11:19:15 -08:00
3e897df51c
Update supports_buft and supports_op for quantized models
2026-01-15 11:19:15 -08:00
56d596775d
Change openvino device_type to GPU; Enable flash_attn
2026-01-15 11:19:15 -08:00
65e1b1af6d
Fix after rebasing
...
- Layout of cache k and cache v are unified: [seq, n_head, head_size]
- Add CPY and FLASH_ATTN_EXT, flash attn is not used yet
- Skip test-backend-ops due to flash attn test crash
- Add mutex around graph conversion to avoid test-thread-safety fali in the future
- Update NPU config
- Update GPU config to disable SDPA opt to make phi-3 run
2026-01-15 11:19:15 -08:00
14c8a85c32
Perf: RMS fused to OV internal RMS op
2026-01-15 11:19:15 -08:00
a7b611bc93
Minor updates for raising PR
2026-01-15 11:19:15 -08:00
f4123be967
Fix test-backend-ops
2026-01-15 11:19:15 -08:00
839f8c66a0
Remove CPY
2026-01-15 11:19:15 -08:00
7bda5021f9
Fix NPU
2026-01-15 11:19:15 -08:00
63d000ba40
Support op SET_ROWS
2026-01-15 11:19:15 -08:00
9a91ca6ef9
Optimize tensor conversion, improve TTFT
2026-01-15 11:19:15 -08:00
37ff226bb6
Use CiD for NPU
2026-01-15 11:19:15 -08:00
1141350310
Skip test-thread-safety; Run ctest only in ci/run.sh
2026-01-15 11:18:28 -08:00
fc865340d5
Fix test-backend-ops
2026-01-15 10:26:28 -08:00
2f99135ccc
Update build.md
2026-01-15 10:26:28 -08:00
43489bbfaa
Revert changes in fuse_to_sdpa
2026-01-15 10:26:28 -08:00
1a19566b23
add mark decomp pass
2026-01-15 10:26:28 -08:00
93b2d09a2d
mulmat type conversion update
2026-01-15 10:26:28 -08:00
e2fdc1b988
mulmat input conversion fix
2026-01-15 10:26:28 -08:00
01cdf4a9cc
matmul in fp32
2026-01-15 10:26:28 -08:00
9cf56d6837
temp. changes for mark decomp
2026-01-15 10:26:28 -08:00
4e7f04a307
Fix llama-perplexity
2026-01-15 10:26:28 -08:00
75eec6265f
Fix llama-bench; Clang-format
2026-01-15 10:26:28 -08:00
6dc4b90635
Fix NPU
2026-01-15 10:26:28 -08:00
44f4cf34b1
Fix Phi3 ROPE; Add test-backend-ops
2026-01-15 10:26:28 -08:00
1ed49bbfaf
Fix llama-cli
2026-01-15 10:26:28 -08:00
ea75772e48
Added OpenVINO CI/CD. Updated docs
2026-01-15 10:26:25 -08:00
d61f83c9b7
Fix CPY due to cgraph change
2026-01-15 10:23:35 -08:00
f3c0519096
Reduce memory: free ov weights node after graph conversion
2026-01-15 10:20:18 -08:00
a80da69448
Pull out sin cos from rope
2026-01-15 10:20:18 -08:00
3533c14cf6
Fix Phi3 SwiGLU and SoftMax
2026-01-15 10:20:18 -08:00
0fa7a5efef
Refactor: remove past_token_len from extra_inputs
2026-01-15 10:20:18 -08:00
acf358d1ce
Pull out indices creation for kv cache update
2026-01-15 10:20:18 -08:00
bf5414c95e
Replace Concat with Broadcast in MulMat for GQA
2026-01-15 10:20:18 -08:00
ebc4fc9f95
Fuse to SDPA
2026-01-15 10:20:18 -08:00
73ee84fffe
Add SwiGLU
2026-01-15 10:20:18 -08:00
4c582ac7a3
Statful transformation for CPU GPU
2026-01-15 10:20:18 -08:00
8afee795ad
Update clang-format
2026-01-15 10:20:18 -08:00
593484ce5f
Refactor: clean, fix warning
2026-01-15 10:20:18 -08:00
42d4240937
Change due to ggml cgraph changes, all device work
2026-01-15 10:20:18 -08:00
e27738a987
Add AMD64 to CMakeLists
2026-01-15 10:20:18 -08:00
592d7f8bbb
Change due to ggml cgraph changes, llama-3.2 CPU work
2026-01-15 10:20:18 -08:00
f7ad77930e
Change due to ggml cgraph changes, not correct yet
2026-01-15 10:20:18 -08:00
d9ca8f5dbe
NPU support version 2: prefill + kvcache
2026-01-15 10:20:18 -08:00
34531abce4
draft NPU support version 2: prefill + kvcache
2026-01-15 10:20:18 -08:00
7fec223334
Add initial NPU support
2026-01-15 10:20:18 -08:00
3051d5ae07
Update openvino build instructions
2026-01-15 10:20:18 -08:00
8ce5cc597a
Add cgraph tensor output name to OV op name
2026-01-15 10:20:18 -08:00
fd324366d0
Update build doc
2026-01-15 10:20:18 -08:00
Yu, Zijun
Ravi Panchumarthy
Cavus Mustafa