2.6 KiB
llama.cpp Vulkan Build on RDNA4 (RX 9070 XT)
What we did
- Chose plain llama.cpp (not Ollama) for local LLM inference — built from source for full control, targeting coding + agent work via Continue.dev.
- Faced a real backend choice: HIP/ROCm (AMD's "official" compute stack) vs. Vulkan. For this exact GPU generation (RDNA4,
gfx1201), current community data showed Vulkan can outperform HIP/ROCm right now, since ROCm kernel support for gfx1201 only landed officially with ROCm 7.2 (March 2026) and is still maturing — the opposite of the usual pattern on older AMD cards. HIP also has a known bug on RDNA4 where the GPU never returns to idle power once the HIP runtime initializes — relevant for a machine meant to stay on 24/7. - Ran a readiness audit before installing anything (kernel driver, Vulkan packages, build tools, disk space, kernel version, group membership) to avoid mid-build surprises.
- Installed missing pieces:
cmake,vulkan-headers,spirv-headers,ninja. - Built with
-DGGML_VULKAN=ON, verified with a smoke-test model — GPU offload confirmed via-ngl 99, ~150–260 t/s generation (well above CPU-only range for the model size used).
Key distinction / Mental model
On brand-new GPU architectures, "the official vendor compute stack" isn't automatically the fastest or most stable choice — driver/kernel maturity for a specific chip can lag behind a more generic API that's had years of broad hardware support baked in. Worth checking current benchmarks per-GPU-generation rather than assuming the "proper" stack wins by default.
Commands / syntax
| Command | What it does |
|---|---|
lspci -k | grep -A3 VGA |
Confirm which kernel driver (e.g. amdgpu) is bound to the GPU |
pacman -Q vulkan-icd-loader vulkan-radeon vulkan-headers shaderc |
Check Vulkan runtime + build stack presence |
cmake -B build -G Ninja -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release |
Configure llama.cpp for a Vulkan-accelerated, optimized, Ninja-driven build |
cmake --build build -j$(nproc) |
Compile using all CPU threads |
./llama-cli -hf <repo> -p "..." -n 50 -ngl 99 |
Quick GPU smoke test — pulls a model from Hugging Face and runs it fully offloaded to GPU |
When to use this
Reference this when setting up llama.cpp (or similar GPU-inference tools) on a new/recent AMD GPU — check current-generation ROCm vs. Vulkan performance data first rather than defaulting to ROCm.
Index line
llama_cpp_vulkan_rx9070xt.md — Building llama.cpp with Vulkan backend on RDNA4 (RX 9070 XT/gfx1201); why Vulkan was chosen over HIP/ROCm for this GPU generation; build commands and smoke-test verification.