happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
llama.cpp/ggml/src
History
Georgi Gerganov b4ae50810e
metal : improve FA + improve MoE (#12612) 
* ggml : FA with different K, V head sizes (CPU)

ggml-ci

* metal : add FA with HS=192

* metal : extend FA to support different K and V head sizes

ggml-ci

* metal : add FA vector kernels for heads K 192 and V 128

ggml-ci

* ggml : restrict op on other backends to equal head sizes

ggml-ci

* metal : optimize FA-vec kernel

ggml-ci

* metal : FA remove mq registers

* metal : improve MoE mul_mat_id condition

ggml-ci

* metal : fix comments + remove unnecessary addition

ggml-ci

* metal : avoid too much shared memory usage with mul_mat_id

ggml-ci
 		2025-03-28 20:21:59 +02:00
..
ggml-blas ggml : add support for dynamic loading of backends (#10469) 2024-11-25 15:13:39 +01:00
ggml-cann [CANN]MUL_MAT optimization (#12382) 2025-03-15 09:31:08 +08:00
ggml-cpu metal : improve FA + improve MoE (#12612) 2025-03-28 20:21:59 +02:00
ggml-cuda metal : improve FA + improve MoE (#12612) 2025-03-28 20:21:59 +02:00
ggml-hip HIP: implement FlashAttention via rocWMMA for CDNA and RDNA3+ (#12032) 2025-03-03 22:10:54 +01:00
ggml-kompute llama : add Qwen2VL support + multimodal RoPE (#10361) 2024-12-14 14:43:46 +02:00
ggml-metal metal : improve FA + improve MoE (#12612) 2025-03-28 20:21:59 +02:00
ggml-musa cuda : enable CUDA Graph on CUDA Toolkit < 12.x (#12394) 2025-03-17 20:25:13 +02:00
ggml-opencl opencl: add multi and vision rope, `gelu_quick` and `im2col` (#12600) 2025-03-27 08:08:08 -07:00
ggml-rpc rpc : send hash when tensor data is above some fixed threshold (#12496) 2025-03-28 08:18:04 +02:00
ggml-sycl SYCL: implement memset ggml backend buffer interface (#12580) 2025-03-27 09:46:00 +08:00
ggml-vulkan metal : improve FA + improve MoE (#12612) 2025-03-28 20:21:59 +02:00
CMakeLists.txt [SYCL] Fix build on Windows when ccache enabled (#9954) (#9976) 2025-03-21 14:58:47 +08:00
ggml-alloc.c ggml : upgrade init_tensor API to return a ggml_status (#11854) 2025-02-28 14:41:47 +01:00
ggml-backend-impl.h ggml : upgrade init_tensor API to return a ggml_status (#11854) 2025-02-28 14:41:47 +01:00
ggml-backend-reg.cpp ggml-backend : fix backend search path (#12330) 2025-03-11 14:25:17 +01:00
ggml-backend.cpp ggml : portability fixes for VS 2017 (#12150) 2025-03-04 18:53:26 +02:00
ggml-common.h CUDA: use arch list for compatibility check (#11775) 2025-02-11 00:17:22 +01:00
ggml-impl.h ggml : riscv: add 128-bit RVV support (#12530) 2025-03-27 08:38:34 +02:00
ggml-opt.cpp ggml-opt: fix data corruption (ggml/1022) 2024-11-21 09:22:02 +02:00
ggml-quants.c ggml : portability fixes for VS 2017 (#12150) 2025-03-04 18:53:26 +02:00
ggml-quants.h ggml : build backends as libraries (#10256) 2024-11-14 18:04:35 +01:00
ggml-threading.cpp ggml : build backends as libraries (#10256) 2024-11-14 18:04:35 +01:00
ggml-threading.h remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (#10797) 2024-12-12 19:02:49 +01:00
ggml.c metal : improve FA + improve MoE (#12612) 2025-03-28 20:21:59 +02:00
gguf.cpp cmake : add sanitizer flags for llama.cpp (#11279) 2025-01-18 16:18:15 +02:00