5 Commits

Author	SHA1	Message	Date
Johannes Gäßler	bdcb8f4222	CUDA: int8 tensor cores for MMQ (q4_K, q5_K, q6_K) (#7860)	2024-06-11 08:26:07 +02:00
Johannes Gäßler	1f0dabda8d	CUDA: use tensor cores for MMQ (#7676) ... * CUDA: int8 tensor cores for MMQ (legacy quants) * fix out-of-bounds writes * __builtin_assume -> GGML_CUDA_ASSUME * fix writeback returning too early	2024-06-10 11:45:13 +02:00
Johannes Gäßler	42b53d192f	CUDA: revise q8_1 data layout for mul_mat_q (#7824)	2024-06-09 09:42:25 +02:00
Johannes Gäßler	7d1a378b8f	CUDA: refactor mmq, dmmv, mmvq (#7716) ... * CUDA: refactor mmq, dmmv, mmvq * fix out-of-bounds write * struct for qk, qr, qi * fix cmake build * mmq_type_traits	2024-06-05 16:53:00 +02:00
slaren	ae1f211ce2	cuda : refactor into multiple files (#6269)	2024-03-25 13:50:23 +01:00