Repository navigation
docs(support): document the Vulkan GPU backend (#1029) - #1033
Conversation
The GPU support section stated that llamafile has no Vulkan backend, but Vulkan has been supported since mozilla-ai#892 (Feb 2026): llamafile/vulkan.c loads a prebuilt ggml-vulkan.so/.dll through the same shared probe/register path as CUDA and ROCm, llamafile_gpu_parse() accepts --gpu vulkan (alias vk), and llamafile/release.sh bundles ggml-vulkan.so and ggml-vulkan.dll into the full llamafile release. Correct the intro sentence, describe how the Vulkan library is located and built, add a Vulkan row to the support table, split the Intel row so it refers to the missing oneAPI/SYCL backend rather than to Vulkan, and list --gpu vulkan alongside --gpu nvidia/--gpu amd in the verification tips. Fixes mozilla-ai#1029 Assisted-by: Claude <noreply@anthropic.com> Signed-off-by: Tai An <antai12232931@outlook.com>
367465b to
82d3f9e
Compare
| Vulkan is loaded the same way as CUDA and ROCm: llamafile looks for a | ||
| prebuilt `ggml-vulkan.so` / `ggml-vulkan.dll` (the full `llamafile` release | ||
| bundles both) and otherwise falls back to a library you build yourself with | ||
| `llamafile/vulkan.sh` (`vulkan.bat` on Windows) — see |
There was a problem hiding this comment.
This implies the llamafile-thin behaviour requires a user to manually intervene to ensure the backend library is built? I am not a user of llamafile and do not have a system spare that I can verify this.
My understanding was this was fully transparent to the user with llamafile-thin, assuming their system has the ability to build the backend support. The details of how this is done is described in the technical details page of the docs. As such I'm not sure if this paragraph is relevant here.
| | AMD | HIP / rocBLAS | Linux, Windows | Supported | Pass `-ngl 999` to offload; multi-GPU may be broken on Radeon (see below) | | ||
| | Intel / other | — | — | Not supported | No Vulkan or SYCL backend; runs on CPU. For these, build llama.cpp directly | | ||
| | Any (incl. Intel) | Vulkan | Linux, Windows, macOS | Supported | Pass `-ngl 999` to offload; select with `--gpu vulkan`. Used in `--gpu auto` when no vendor backend is available | | ||
| | Intel / other | oneAPI / SYCL | — | Not supported | No SYCL backend; use the Vulkan row above, or build llama.cpp directly | |
There was a problem hiding this comment.
Since there's no plans to support this API AFAIK, it's probably sufficient to remove this row. If there's actual demand for it requested over the Vulkan backend by enough users, or interested users asking about such support, then there may be more value in keeping such in documentation.
The previous engagement for support was before Vulkan backend became available.
Addresses review feedback on mozilla-ai#1033.
|
Thanks — dropped the SYCL row in dcb1420 (the prose above the table still notes there's no oneAPI/SYCL backend, so the information isn't lost, but the table no longer advertises an unsupported target). On the paragraph: I checked That said, your point that the how belongs in the technical docs is fair, so I reworded it to just state the lookup order and defer to Building the GPU libraries rather than naming the build script here. |
Problem
The GPU support section — which I contributed in #997 — says:
That is wrong about Vulkan, as @polarathene points out in #1029. Vulkan landed in #892 (Feb 2026) and is a first-class backend today. My PR repeated a stale assumption; this corrects it.
What the code actually does
llamafile/vulkan.cloads a prebuiltggml-vulkan.so/ggml-vulkan.dlland registers it throughgpu_backend.c— the same load → suppress-logs → device-count-gate → register path as CUDA and ROCm.gpu_backend.hnames Vulkan explicitly as one of the prebuilt-DSO backends.llamafile_gpu_parse()inllamafile/llamafile.caccepts--gpu vulkan(aliasvk), andllamafile_has_vulkan()participates in--gpu autodetection.llamafile/release.shbundles bothggml-vulkan.soandggml-vulkan.dllinto the fullllamafilerelease; otherwisellamafile/vulkan.sh/vulkan.batbuild it (already documented indocs/building_dlls.md).ImportVulkanImpl()deliberately does not skip Apple Silicon, unlike CUDA.Changes (docs only, single file)
--gpu vulkan), and split the old "Intel / other" row so its "Not supported" applies to oneAPI/SYCL rather than to Vulkan — it now points readers at the Vulkan row.--gpu vulkannext to--gpu nvidia/--gpu amd.I deliberately kept this to facts I could verify in the tree. The open questions from the #1023 thread (prebuilt arch/libc coverage, exact fallback ordering,
.zipartifact contents) are not asserted here.Fixes #1029
🤖 Generated with Claude Code