[niehen6174]

Motivation

Community ComfyUI-GGUF exports of Qwen-Image-2.1 (e.g. abenzerps/Qwen-Image-2.1-Uncensored-GGUF) fail to load. Unlike the native-name exports validated earlier, they also quantize the plain nn.Linear layers outside the blocks (modulation, time_text_embed, norm_out, proj_out), which have no packed-weight parameter to load into.

Modifications

• Dequantize those layers to BF16 on load through a new gguf_dequantize_prefixes on the DiT, the same mechanism the GGUF text-encoder path uses.
• packed_qkv_weight skips quantized projections instead of reading .weight, which GGUF layers do not have.
• CPU unit tests for both fixes (they fail on main), plus a GGUF docs/cookbook validated-scope update.

sglang generate --model-path Qwen/Qwen-Image-2.1 \
  --component-weights-paths.transformer abenzerps/Qwen-Image-2.1-Uncensored-GGUF/qwen-image-2.1-UC-Q4_K_M.gguf \
  --prompt "..." --save-output

Accuracy Tests

1x RTX 5090, default sampling params (1024x1024, 40 steps, guidance 1.0), seed 42, default auto memory policy.

Transformer	PSNR vs official BF16 (fox prompt)
qwen-image-2.1-Q8_0.gguf (base revision, plain quantization)	38.8 dB
qwen-image-2.1-UC-Q4_K_M.gguf	23.8 dB

The UC checkpoint is not a pure quantization. A tensor-level diff of its BF16 export against the official weights shows that every non-attention tensor is bit-identical. The attention to_q/k/v/out weights equal bf16(W + 1.0 * B @ A) using the rank-16 LoRA shipped in the same repo, which explains the lower PSNR. Visually, the UC Q4_K_M output keeps the composition, subject, text rendering, and lighting of the official model on all three prompts tested:

Speed Tests and Profiling

Same setup as above, main + this PR:

Transformer	DiT weights	Peak VRAM	Denoise s/step
Official BF16	14.2 GB	17.0 GB	0.43-0.48
GGUF Q8_0	7.6 GB	11.1 GB	0.47
GGUF Q4_K_M	4.6 GB	8.25 GB	0.47

As the GGUF guide notes, GGUF is a capacity option: the packed linears are dequantized each step and then run the native GEMM, so per-step latency is slightly higher.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Not covered: TP, cache-dit, editing / RGBA with these files, other GGUF types (Q4_0/Q5_K_M/Q6_K), and the FP8 / INT8-ConvRot / NVFP4 safetensors in the same repo.

---

CI States

Latest PR Test (Base): ✅ Run #37465558588
Latest PR Test (Extra): ⚠️ Not enabled -- add run-ci-extra label to opt in.
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

[mickqian]

/tag-and-rerun-ci