e459796110
Fixes two critical issues with Metal backend on AMD/Intel discrete GPUs: 1. Auto-disable concurrent dispatch on non-Apple GPUs - MTLDispatchTypeConcurrent has broken memory barriers on AMD - Causes L2 cache coherency issues leading to corrupted output - Now auto-detects via supports_gpu_family_apple7 and uses serial dispatch 2. Use Managed buffer mode on discrete GPUs for cached PCIe reads - Shared mode uses uncached PCIe reads (~3 MB/s) on discrete GPUs - Managed mode enables cached reads (~3 GB/s) with explicit sync - Adds didModifyRange after CPU writes, synchronizeResource for reads - Improves generation speed from ~11 t/s to ~60 t/s on AMD Radeon Pro 5300M Performance on AMD Radeon Pro 5300M: - Before: 11.5 t/s generation (corrupted output with concurrent dispatch) - After: 60+ t/s generation (correct output, 88% of Vulkan performance) Environment variable overrides for testing: - GGML_METAL_MANAGED_BUFFERS_DISABLE/ENABLE - GGML_METAL_CONCURRENCY_DISABLE (still works) Also adds bench_metal.sh with benchmark and conversation modes. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| cmake | ||
| include | ||
| src | ||
| .gitignore | ||
| CMakeLists.txt | ||