26 lines
No EOL
2.6 KiB
Markdown
26 lines
No EOL
2.6 KiB
Markdown
## llama.cpp Vulkan Build on RDNA4 (RX 9070 XT)
|
||
|
||
### What we did
|
||
- Chose plain llama.cpp (not Ollama) for local LLM inference — built from source for full control, targeting coding + agent work via Continue.dev.
|
||
- Faced a real backend choice: HIP/ROCm (AMD's "official" compute stack) vs. Vulkan. For this exact GPU generation (RDNA4, `gfx1201`), current community data showed Vulkan can *outperform* HIP/ROCm right now, since ROCm kernel support for gfx1201 only landed officially with ROCm 7.2 (March 2026) and is still maturing — the opposite of the usual pattern on older AMD cards. HIP also has a known bug on RDNA4 where the GPU never returns to idle power once the HIP runtime initializes — relevant for a machine meant to stay on 24/7.
|
||
- Ran a readiness audit *before* installing anything (kernel driver, Vulkan packages, build tools, disk space, kernel version, group membership) to avoid mid-build surprises.
|
||
- Installed missing pieces: `cmake`, `vulkan-headers`, `spirv-headers`, `ninja`.
|
||
- Built with `-DGGML_VULKAN=ON`, verified with a smoke-test model — GPU offload confirmed via `-ngl 99`, ~150–260 t/s generation (well above CPU-only range for the model size used).
|
||
|
||
### Key distinction / Mental model
|
||
On brand-new GPU architectures, "the official vendor compute stack" isn't automatically the fastest or most stable choice — driver/kernel maturity for a *specific chip* can lag behind a more generic API that's had years of broad hardware support baked in. Worth checking current benchmarks per-GPU-generation rather than assuming the "proper" stack wins by default.
|
||
|
||
### Commands / syntax
|
||
| Command | What it does |
|
||
|---|---|
|
||
| `lspci -k \| grep -A3 VGA` | Confirm which kernel driver (e.g. `amdgpu`) is bound to the GPU |
|
||
| `pacman -Q vulkan-icd-loader vulkan-radeon vulkan-headers shaderc` | Check Vulkan runtime + build stack presence |
|
||
| `cmake -B build -G Ninja -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release` | Configure llama.cpp for a Vulkan-accelerated, optimized, Ninja-driven build |
|
||
| `cmake --build build -j$(nproc)` | Compile using all CPU threads |
|
||
| `./llama-cli -hf <repo> -p "..." -n 50 -ngl 99` | Quick GPU smoke test — pulls a model from Hugging Face and runs it fully offloaded to GPU |
|
||
|
||
### When to use this
|
||
Reference this when setting up llama.cpp (or similar GPU-inference tools) on a new/recent AMD GPU — check current-generation ROCm vs. Vulkan performance data first rather than defaulting to ROCm.
|
||
|
||
### Index line
|
||
`llama_cpp_vulkan_rx9070xt.md` — Building llama.cpp with Vulkan backend on RDNA4 (RX 9070 XT/gfx1201); why Vulkan was chosen over HIP/ROCm for this GPU generation; build commands and smoke-test verification. |