Meta’s Muse Glimmer downloads as a 17 GB GGUF and generates at about 1.6 tokens per second on a four core laptop CPU. That single number decides how you read the rest of this page, because it is the difference between a model you chat with and a model you queue jobs against.
Muse Glimmer is Meta’s 30B open-weight multimodal model, released under the Apache License 2.0 and distilled from Muse Spark, Meta’s model for agentic and coding work. It reads text and images, it reasons before it answers, and it carries a 131072 token native context. Unlike the Muse Spark models, which exist only behind Meta Model API, these weights are yours to download: no key, no account, no region gate. Everything below was run on Ubuntu 26.04.1 LTS with a source build of llama.cpp at commit f3f1a8f in September 2026, against the published Q4_K_M checkpoint on 8 vCPU with no GPU.
What the download actually weighs
Four files sit in the GGUF repository and you do not need all of them. Sizes below are the byte counts from the repository’s own LFS metadata, matched against what landed on disk:
| File | Bytes | What it is | Needed? |
|---|---|---|---|
Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf | 16,756,683,904 | Text model, K-quant | Yes |
Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf | 19,653,960,832 | Text model, dynamic K-quant | Alternative |
mmproj-Muse-Glimmer-30B-Q4_K_M.gguf | 1,400,328,928 | Vision projector | Only for image input |
dflash-Muse-Glimmer-30B-Q4_K_M.gguf | 1,631,208,128 | DFlash speculative draft model | Optional |
Meta publishes accuracy costs for both quants against full precision, measured across 15 benchmarks: 0.2 percent degradation for the dynamic K-quant, 1.0 percent for the 17 GB build. Paying 2.9 GB to recover 0.8 percentage points is a reasonable trade if you have the RAM for it.
Full precision is not published as GGUF. The BF16 safetensors checkpoint totals 59,553,253,376 bytes, which is 59.6 GB, and that route runs through vLLM rather than llama.cpp.
One number worth getting right before you quote it anywhere. The safetensors repository reports 29,776,626,688 parameters and Meta calls the model 30B, but llama.cpp reads the text checkpoint and prints something smaller:
print_info: model params = 27.85 B
print_info: arch = muse-glimmer
print_info: file type = Q4_K - MediumBoth figures are correct and they describe different things. The safetensors index carries the language model plus a vision tower, adapter and projection; the GGUF text file carries only the language model, and the perception encoder ships separately as that 1.4 GB mmproj. The difference works out at roughly 1.9B parameters, which matches the ViT-G/14 encoder described on the model card. So 27.85B is what generates your tokens, and the image encoder is what you bolt on when you want vision.
Check your RAM before you start
The model file is 16.8 GB, which tempts people into sizing a 16 GB box. It is not enough, and the reason is more interesting than the KV cache. llama.cpp maps the quantized weights and then builds a repacked copy for CPU inference, so both live in memory at once:
load_tensors: CPU_Mapped model buffer size = 15967.91 MiB
load_tensors: CPU_REPACK model buffer size = 11621.39 MiB
sched_reserve: CPU compute buffer size = 157.02 MiBRead those two buffers differently. The 15.6 GiB mapped buffer is file backed, so the kernel can reclaim it under pressure and it inflates resident size without being a hard requirement. The 11.3 GiB repack buffer is private memory and is not negotiable; it is the bulk of the 15.75 GB that free reports as used. Total resident size for the server process measured 28.7 GB with a 131072 token context and text only, and 29.7 GB with the vision projector also loaded, so treat published RSS figures for this model, including ours, as an upper bound rather than the working set.
Do not trust the automatic fit either. On the same run llama.cpp reported this before loading a single tensor:
common_params_fit_impl: projected to use 4529 MiB of host memory vs. 32090 MiB of total host memory
common_fit_params: successfully fit params to free device memoryIt projected 4.4 GiB and the process finished around 28 GB. The fit step declares success either way, so it is no protection against choosing a machine that is too small.
For sizing, use Meta’s own target hardware table rather than anyone’s RSS reading. The card pairs full precision with 64 GB of VRAM, the dynamic K-quant with 32 GB, and the 17 GB K-quant with 24 GB, and it explains the budget: under 20 GB of weights leaves headroom for the KV cache, the perception encoder and the speculative decoding drafter inside a 24 GB or 32 GB envelope. Our 32 GB CPU host ran the 17 GB build with the full context and no swap device configured at all, so nothing paged out at any point. With the model loaded, free reported 15.75 GB used against 16.4 GB in buffers and cache, which is the mapped-versus-private split above showing up as two separate numbers.
The architecture behind the modest KV cost is readable straight out of the file rather than taken on trust:
print_info: n_ctx_train = 131072
print_info: n_layer = 52
print_info: n_head = 32
print_info: n_head_kv = 2
print_info: n_swa = 2048
print_info: is_swa_any = 1
print_info: n_expert = 0
print_info: n_vocab = 202048n_expert = 0 is the line that matters if you were expecting a mixture of experts. This is a dense 52 layer model, so every parameter is active on every token and there is no sparse routing to cut the compute. Two KV heads and a 2048 token sliding window are what keep the cache affordable at 131072.
Download the GGUF weights
Pull only the files you need. The text model plus the vision projector covers everything in this guide:
pip install -U huggingface_hub
hf download meta-models/Muse-Glimmer-30B-GGUF --local-dir ./muse-glimmer \
--include "Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf" \
--include "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf"On Ubuntu 26.04 that pip install refuses to touch the system interpreter, because the distro marks it externally managed. Use a virtual environment, or skip the Python client entirely and take the files over HTTP, which is what we did here so the transfer could resume across a link that kept dropping:
B=https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF/resolve/main
curl -L -C - --speed-limit 200000 --speed-time 30 \
-o Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
"$B/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf"The --speed-limit and --speed-time pair is the useful part on a 17 GB transfer. Without them a stalled connection hangs indefinitely instead of erroring out, and -C - then has nothing to resume from. Check the byte count against the table above when it finishes, because a truncated GGUF fails at load with a confusing tensor error rather than an honest complaint about size.
Build llama.cpp with Muse Glimmer support
Muse Glimmer support landed upstream in release b10353, which added the muse-glimmer architecture, the vision projector and the ATEM tool call parser. DFlash speculative decoding was already there, having landed separately in June 2026; the Muse Glimmer merge only extended it to multimodal batches. Anything older than b10353 refuses the checkpoint outright. Prebuilt binaries cover Linux, macOS and Windows on the releases page, and a source build takes a few minutes:
sudo apt install -y cmake build-essential git libcurl4-openssl-dev
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF \
-DLLAMA_BUILD_UI=OFF -DLLAMA_USE_PREBUILT_UI=OFF
cmake --build build -j"$(nproc)" --target llama-serverThe two UI flags keep the browser interface out of the build. That step pulls assets over the network, and skipping it avoids a dependency on npm you do not otherwise need. The HTTP API is unaffected either way.
Confirm the checkout actually knows the architecture before you spend fifteen minutes debugging a load failure:
grep -c LLM_ARCH_MUSE_GLIMMER src/llama-arch.cppA 1 means you are fine and a 0 means the checkout predates support. That grep is more reliable than the version string, and here is why. The build number comes from the commit count, so a shallow clone reports nonsense. The same commit built two ways gave two different answers:
version: 0.4.0-dev (build 200, commit f3f1a8f) # source build, git clone --depth 200
version: 0.4.0-dev (build 10867, commit f3f1a8f27) # official prebuilt, same commitIdentical code, and one of them looks thousands of builds too old to work. If you clone shallow, trust the grep.
Start the server
The command below is what produced every measurement on this page. Drop --mmproj if you only want text:
./build/bin/llama-server \
-m ./muse-glimmer/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
--mmproj ./muse-glimmer/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
-a muse-glimmer \
-c 131072 -np 1 -t 8 \
--host 127.0.0.1 --port 8080 --api-key your-key-here \
--jinjaWorth knowing what those actually do, because two of them are less load bearing than they look. --jinja applies the chat template embedded in the GGUF, and it is enabled by default on current builds, so passing it is belt and braces rather than a requirement. -a muse-glimmer is cosmetic: it sets the id reported by /v1/models and echoed back in responses. Sending a deliberately wrong model name proved the point: the request was accepted and the response echoed muse-glimmer back instead of erroring, because single model mode never validates that field. -t 8 is the one that matters on CPU, pinning threads to the vCPU count.
Do not pass --chat-template-file. Upstream does ship models/templates/muse-glimmer.jinja, but --jinja already uses the copy inside the GGUF, and llama.cpp selects the ATEM tool call parser by recognising that template’s control tokens. Overriding it is how you silently lose tool calling.
The startup log confirms the context you actually got, which is not always the one you asked for:
srv load_model: loading model '/home/user/muse-glimmer/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf'
srv load_model: loaded multimodal model, '/home/user/muse-glimmer/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf'
srv load_model: initializing, n_slots = 1, n_ctx_slot = 131072, kv_unified = 'false'
srv llama_server: model loaded
srv llama_server: listening on http://127.0.0.1:8080
srv llama_server: NOTICE: server default port will be changed to :9931 in a future release
srv llama_server: ref: https://github.com/ggml-org/llama.cpp/pull/26508The full 131072 came through with no capping, and the server was listening 11.4 seconds after launch. Read n_ctx_slot rather than your own -c value. With an explicit -np N and kv_unified = 'false', as above, the context divides between slots, so -np 4 against -c 131072 leaves each slot only 32768. Leave -np off and the default takes a different path, reporting four slots with kv_unified = 'true' and the full context available to each, so check the log rather than assuming either behaviour.
That port notice is worth acting on. It fires whenever you are on 8080, and the default moves to 9931 in a later release, so pin --port in anything you automate.
Measure what it actually generates
All figures here come from the server’s own timings block on an 8 vCPU guest backed by an Intel Core i7-1165G7, which is four physical cores with SMT, no GPU. Context 8192, temperature 0, seed 42, reasoning_strength: low, same prompt three times per configuration:
| Configuration | Run 1 | Run 2 | Run 3 | Mean | Tokens generated |
|---|---|---|---|---|---|
| Baseline, no draft model | 1.552 | 1.600 | 1.526 | 1.559 tok/s | 123 |
| DFlash speculative decoding | 1.726 | 1.622 | 1.691 | 1.680 tok/s | 131 |
Speculative decoding bought 7.7 percent, at a draft acceptance rate of 83 accepted out of 192 drafted, or 43.2 percent. Enable it by adding the draft model:
-md ./muse-glimmer/dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
--spec-type draft-dflash --spec-draft-n-max 4The startup log prints the draft configuration it settled on, which is the fastest way to confirm the draft model was picked up rather than silently ignored:
common_speculative_impl_draft_dflash: adding speculative implementation 'draft-dflash'
common_speculative_impl_draft_dflash: - n_max=4, n_min=0, p_min=0.00
common_speculative_impl_draft_dflash: - block_size=16, mask_token_id=201818, n_extract=5, sample_from_anchor=trueTwo caveats on that comparison. The two rows are not the same output: at temperature 0 the baseline answered in 123 tokens and the speculative run in 131, because different batch sizes take different kernels and produce small numerical differences, which upstream treats as expected behaviour. And temperature 0 is itself off spec, since Meta recommends 1.0 with top_p 0.95 and top_k 64. We fixed it at 0 for reproducibility, which is the right call for benchmarking and the wrong one for output quality.
Set expectations honestly on that 7.7 percent. Speculative decoding trades spare parallel compute for fewer sequential steps, and a CPU that is already compute bound has little spare to trade. The 1.6 GB the draft model costs in RAM is a fair price at these numbers, but it is not the multiple the same technique delivers on a GPU, where Meta reports 3.1x on an RTX 5090.
The practical reading: at 1.6 tokens per second a 130 token answer takes about 80 seconds, and this model thinks before it answers, so the real wait is longer than the visible reply suggests. That is a background worker, not an interactive assistant. If you want conversational latency, the same weights want a GPU and either vLLM or SGLang instead.
Control how much it thinks
Muse Glimmer reasons by default and the template defaults to high. Four levels exist: low, medium, high and xhigh, and Meta recommends the top two for coding and agentic work. Three different mechanisms set it, and they all reach the same control:
--chat-template-kwargs '{"reasoning_strength":"low"}' # server wide
{"chat_template_kwargs":{"reasoning_strength":"low"}} # per request
{"reasoning_effort":"low"} # per request, OpenAI spellingThe OpenAI spelling works. Sending reasoning_effort produced token counts identical to the reasoning_strength runs, 42 completion tokens and 109 reasoning characters at low, 69 and 223 at high. The chat template does the aliasing itself, rewriting “Reasoning effort” to “Reasoning strength” before it renders. Meta’s own llama.cpp guide says reasoning_effort is unimplemented, which was true when written and is not true on a current build.
The cost difference is not subtle. Same prompt, same temperature, same correct answer of 391, single run each:
| Setting | Completion tokens | Generation time | Answer |
|---|---|---|---|
low | 42 | 22.8 s | 391 |
high (default) | 69 | 46.1 s | 391 |
The default took roughly twice as long to reach the same three characters. Those are single runs, so read the token counts, which are deterministic here, ahead of the wall clock times, which carry the same few percent of noise as the table above. Lower is not automatically worse, and where the caller can validate or retry cheaply the extra thinking buys very little.
What you cannot currently do is switch reasoning off. The flag help for --reasoning-budget offers “0 for immediate end”, and reasoning_effort: "none" is meant to disable thinking outright, so we tried both against this model. Neither stopped it. --reasoning-budget 0 returned 69 completion tokens and 223 reasoning characters, the same counts high produces, so it was ignored. reasoning_effort: "none" came back at 60 tokens and 188 reasoning characters, distinct from both low and high, and we cannot account for that number.
There is a mechanical reason for the budget being inert, and it is worth knowing because it dates the behaviour. Budget enforcement needs the thinking start and end tags to be registered for the chat format, and llama.cpp has an open pull request to set them for Muse Glimmer precisely because the reasoning budget is not being applied. Nothing in the template offers an off switch either: it carries no enable_thinking flag, and instead injects a Reasoning strength: <level> line into the system block, defaulting to high. So low is the real floor for now, and this is the one section of this page most likely to age.
By default the thinking arrives in message.reasoning_content and message.content stays clean, which is the behaviour you want when parsing responses programmatically. Pass --reasoning-format none to keep it inline instead.
Send it a screenshot
Vision is the reason to carry the extra 1.4 GB. With --mmproj loaded, we fed it this Grafana dashboard and asked it to name the application and the panel titles:
It came back with this, correct down to the breadcrumb and every panel label:
The screenshot is Grafana - Dashboards > Telegraf input plugins.
Panel titles shown:
* Container memory usage (inputs.docker)
* Container CPU percent (inputs.docker)
* Nginx requests and connections (inputs.nginx)
* PostgreSQL backends and commits (inputs.postgresql)That image cost 1,374 prompt tokens. Prompt processing ran at 4.85 tokens per second and generation at 1.47, so the whole exchange took about six and a half minutes on CPU. Image tokens scale with resolution, so resize before sending and crop to the panel you care about.
Sending the image is fiddlier than the docs suggest. A base64 data URI works, and so does a remote URL, but a local file needs both the file:// scheme and a --media-path directory on the server command line, with the path resolved relative to that directory. Without --media-path both forms fail, and the two failures give different messages, which is covered below.
Errors you will hit
unknown model architecture: ‘muse-glimmer’
Your llama.cpp predates Muse Glimmer support. Reproduced on the official b10344 prebuilt, the last build before the architecture landed:
llama_model_load: error loading model: unknown model architecture: 'muse-glimmer'
llama_model_load_from_file_impl: failed to load model
common_init_: failed to load model '/home/user/muse-glimmer/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf'Upgrade to b10353 or newer, or rebuild from a current checkout. The grep in the build section tells you which side of the line you are on.
bash: /usr/bin/curl: Argument list too long
This comes from your shell, not from llama.cpp, and it is easy to hit on the vision path. The limit is not the two megabyte total that getconf ARG_MAX reports, it is the per-argument cap of 131072 bytes, and base64 encoding a 98 KB image pushes a single JSON argument past it. At 131092 bytes, 20 over the line, bash refuses before curl ever runs:
bash: line 5: /usr/bin/curl: Argument list too longOther shells word it differently, so match on the tail rather than the whole line. Write the payload to a file and reference it:
curl -s http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-key-here" \
-d @payload.jsonfile:// URLs are not allowed unless –media-path is specified
You passed a local image path without starting the server with --media-path. The bare path and the file:// form fail differently, and only the second tells you why:
"/tmp/test-image.png" -> invalid_request_error: Failed to load image or audio file
"file:///tmp/test-image.png" -> invalid_request_error: file:// URLs are not allowed unless --media-path is specifiedAdd --media-path /srv/images to the server command and reference files by a path relative to it, or sidestep the whole thing with a base64 data URI.
request exceeds the available context size
Overrunning the context is not the silent failure it is sometimes described as. The server rejects the request with the numbers spelled out:
exceed_context_size_error: request (200057 tokens) exceeds the available
context size (131072 tokens), try increasing itCompare the token count it reports against the n_ctx_slot from the startup log, not against your -c flag, because -np may have divided it.
Empty content with finish_reason: length
Reasoning tokens count against max_tokens. A request that hits the ceiling while the model is still thinking returns a response whose content is an empty string, with the thinking intact and the answer never written. Asking for an explanation of TCP congestion control with max_tokens: 64 produced exactly that: finish_reason: "length", 289 characters of reasoning and zero characters of answer. Raise max_tokens, or lower the reasoning level, or both.
special_eot_id is not in special_eog_ids
This warning fires at every load, once for the main model and again for the draft model, naming token 200008 <|eot|>. It is ordering noise rather than a real defect: a few lines later the same log prints <|eot|> among the registered end-of-generation tokens. Every request that ran to completion returned finish_reason: "stop", so generation terminates correctly. Ignore it.
exceeds the training context, capping
We did not hit this on the published checkpoint, and every run reported the full 131072. If the server does clamp your slot below what you asked for, the GGUF’s context_length metadata is stale and no serve flag overrides it. Rewrite the key instead, and note that this edits the file in place, so checksum the original first:
python gguf-py/gguf/scripts/gguf_set_metadata.py <model>.gguf muse-glimmer.context_length 131072Where this fits against the other local models
Muse Glimmer’s real appeal is the licence and the footprint rather than a benchmark crown. Meta benchmarks it against Gemma4-31B and Qwen3.6-27B, both open-weight and both comfortable on the same hardware, and on the four multimodal rows of that table Muse Glimmer wins one: Charxiv Reasoning at 78.8 against 77.7 and 78.4. On the other three it trails Qwen3.6-27B by 0.7, 2.0 and 1.0 points, while still beating Gemma4-31B on OmniDocBench v1.5 and MMMU Pro. Treat it as competitive in its size class, not as the best small multimodal model available.
The licence is the genuinely strong part. Apache 2.0 on the weights means no revenue threshold and no monthly-active-user trigger, which is more than can be said for several current open-weight flagships. Read the separate usage policy that ships in the repository as well, because Meta attaches prohibited-use terms alongside the licence rather than inside it.
Against the giant releases the comparison is about fit rather than quality. Kimi K3 and DeepSeek V4 Flash are stronger on text and neither comes close to running on a 32 GB box, while 27.85B dense parameters with an image encoder attached will run on a desktop you already own.
What it is not is fast. At 1.6 tokens per second the honest use cases are batch classification, screenshot and document extraction, and overnight jobs where a six minute answer is fine. For anything a person waits on, put the weights on a GPU. If you would rather skip the build entirely, Ollama publishes the model as 15 tags, from an 18 GB default through 65 GB BF16 and MLX builds, with several dflash variants in between. Our open source LLM comparison tracks how the licences and download sizes stack up across the current field, and running a local LLM with llama.cpp covers the server flags in more general terms.
