happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
llama.cpp/ggml/src/ggml-sycl
History
Matthew Michel 9de9672adb
sycl: use async memory allocation to fix crashes during graph recording (#16644) 
* sycl: use async memory allocation to fix graph recording failures

GGML_SYCL_DISABLE_GRAPHS=0 causes crashes because:
  - Host waits are currently unsupported in graph recording mode.
  - SYCL malloc / free calls are unsupported in graph recording mode.

The following changes are made to fix SYCL graph functionality:
  - When graphs are enabled, use the SYCL async memory extension for temp
    buffers which is supported with SYCL graphs.
  - For compiler versions that do not support this extension, skip
    graphs with the affected op.
  - Switch from USM shared to device memory as the async extension
    currently just supports device allocations.

* Address reviewer feedback

* Use global async variable to decide path in sycl_ext_[malloc_device|free]
 		2025-10-23 09:05:15 +08:00
..
dpct [SYCL] refactor soft_max, add soft_max_back (#16472) 2025-10-09 10:25:11 +03:00
CMakeLists.txt SYCL: Bump oneMath commit (#14152) 2025-06-13 08:45:37 +01:00
backend.hpp sycl : add PAD_REFLECT_D1 operator support (#16145) 2025-10-21 00:21:12 +02:00
binbcast.cpp [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 2025-10-12 21:53:35 +08:00
binbcast.hpp [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 2025-10-12 21:53:35 +08:00
common.cpp SYCL: Remove misleading ggml_sycl_op_flatten function (#12387) 2025-03-31 11:25:24 +02:00
common.hpp [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 2025-10-12 21:53:35 +08:00
concat.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
concat.hpp SYCL: Refactor ggml_sycl_compute_forward (#11121) 2025-01-10 08:13:03 +08:00
conv.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
conv.hpp SYCL: Refactor ggml_sycl_compute_forward (#11121) 2025-01-10 08:13:03 +08:00
convert.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
convert.hpp sycl: addressing non-contiguous src1 mul_mats (nc and batched) (#13343) 2025-05-08 10:08:01 +01:00
count-equal.cpp [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 2025-10-12 21:53:35 +08:00
count-equal.hpp [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 2025-10-12 21:53:35 +08:00
cpy.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
cpy.hpp SYCL: Add set_rows support for quantized types (#14883) 2025-07-28 20:32:15 +05:30
dequantize.hpp sycl: Add reorder to Q6_K mmvq implementation (#13885) 2025-06-09 11:47:07 +02:00
dmmv.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
dmmv.hpp llama : reorganize source code + improve CMake (#8006) 2024-06-26 18:33:02 +03:00
element_wise.cpp SYCL: Add support for FLOOR,CEIL,ROUND and TRUNC unary operators (#16613) 2025-10-20 11:08:32 +03:00
element_wise.hpp SYCL: Add support for FLOOR,CEIL,ROUND and TRUNC unary operators (#16613) 2025-10-20 11:08:32 +03:00
gemm.hpp sycl: Batched mulmat rework for oneDNN dispatch (#14617) 2025-07-14 10:37:35 +01:00
getrows.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
getrows.hpp SYCL: Remove misleading ggml_sycl_op_flatten function (#12387) 2025-03-31 11:25:24 +02:00
ggml-sycl.cpp sycl: use async memory allocation to fix crashes during graph recording (#16644) 2025-10-23 09:05:15 +08:00
gla.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
gla.hpp SYCL: Add gated linear attention kernel (#11175) 2025-01-15 11:20:17 +08:00
im2col.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
im2col.hpp SYCL: Remove misleading ggml_sycl_op_flatten function (#12387) 2025-03-31 11:25:24 +02:00
mmq.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
mmq.hpp llama : reorganize source code + improve CMake (#8006) 2024-06-26 18:33:02 +03:00
mmvq.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
mmvq.hpp llama : reorganize source code + improve CMake (#8006) 2024-06-26 18:33:02 +03:00
norm.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
norm.hpp SYCL: Remove misleading ggml_sycl_op_flatten function (#12387) 2025-03-31 11:25:24 +02:00
outprod.cpp sycl: Add more debug prints (#13640) 2025-05-26 10:28:53 +02:00
outprod.hpp SYCL: Refactor ggml_sycl_compute_forward (#11121) 2025-01-10 08:13:03 +08:00
pad.cpp [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 2025-10-12 21:53:35 +08:00
pad.hpp [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 2025-10-12 21:53:35 +08:00
pad_reflect_1d.cpp sycl : add PAD_REFLECT_D1 operator support (#16145) 2025-10-21 00:21:12 +02:00
pad_reflect_1d.hpp sycl : add PAD_REFLECT_D1 operator support (#16145) 2025-10-21 00:21:12 +02:00
presets.hpp SYCL SET operator optimized for F32 tensors (#16350) 2025-10-17 10:36:40 +08:00
quantize.hpp sycl: refactor quantization to q8_1 (#14815) 2025-07-28 11:05:53 +01:00
quants.hpp sycl: fixed semantics of block offset calculation (#14814) 2025-07-24 11:09:57 +01:00
rope.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
rope.hpp SYCL: Add non-contiguous support in ROPE (#12993) 2025-04-21 19:13:30 +05:30
set.cpp SYCL SET operator optimized for F32 tensors (#16350) 2025-10-17 10:36:40 +08:00
set.hpp SYCL SET operator optimized for F32 tensors (#16350) 2025-10-17 10:36:40 +08:00
set_rows.cpp ggml : implement set_rows with i32 index (#16159) 2025-09-22 19:13:00 +02:00
set_rows.hpp SYCL: Initial set_rows kernel implementation (#14562) 2025-07-10 09:29:38 +01:00
softmax.cpp [SYCL] refactor soft_max, add soft_max_back (#16472) 2025-10-09 10:25:11 +03:00
softmax.hpp [SYCL] refactor soft_max, add soft_max_back (#16472) 2025-10-09 10:25:11 +03:00
sycl_hw.cpp sycl: GGML_SYCL_DISABLE_OPT on by default for all Intel Devices (#13973) 2025-06-25 18:09:55 +02:00
sycl_hw.hpp sycl: GGML_SYCL_DISABLE_OPT on by default for all Intel Devices (#13973) 2025-06-25 18:09:55 +02:00
tsembd.cpp ggml : fix padding in timestep embedding kernels (#15932) 2025-09-16 15:25:57 +02:00
tsembd.hpp SYCL: Refactor ggml_sycl_compute_forward (#11121) 2025-01-10 08:13:03 +08:00
vecdotq.hpp sycl: fixed semantics of block offset calculation (#14814) 2025-07-24 11:09:57 +01:00
wkv.cpp Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 2025-09-12 09:15:12 +08:00
wkv.hpp llama: Add support for RWKV v7 architecture (#12412) 2025-03-18 07:27:50 +08:00