Repository navigation
[diffusion] model: support minimax-h3 - #33275
Merged
Merged
Conversation
Add native MiniMax-H3 serving for FL2VA and Ref2VA variants, including text/image/video/audio conditioning, parallel execution, lossless performance paths, quantization and Cache-DiT integration, focused release contracts, and an interactive deployment cookbook. Co-authored-by: zhenaozhenfu <zhenaozhenfu@minimaxi.com> Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: andyluo7 <andy.luo@amd.com> Co-authored-by: Zijie Xia <zijie_xia@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: chao-xue <877184285@qq.com> Co-authored-by: Cursor <cursoragent@cursor.com>
mickqian
requested review from
AgainstEntropy,
BBuf,
DarkSharpness,
Fridge003,
HaiShaw,
HydraQYH,
JustinTong0323,
celve,
ispobock,
merrymercy,
ping1jing2,
sogalin,
wisclmy0611,
yichiche,
yingluosanqian,
yuan-luo and
zijiexia
as code owners
August 2, 2026 10:06
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
github-actions
Bot
added
documentation
Collaborator
Author
|
/tag-and-rerun-ci |
mickqian
changed the title
mickqian
changed the title
2 of 3 tasks
5 tasks done
BBuf
added a commit
to BBuf/sglang
that referenced
this pull request
Aug 5, 2026
…nd reuse a2a staging buffers On the generic USPAttention path (heads at dim=2), the Ulysses input exchange issued three all_to_all_single collectives (Q, K, V), each with a permute+contiguous on both sides, and every USP collective allocated fresh send/recv buffers on each of the 2x40 exchanges per denoise step. - _usp_input_all_to_all_qkv: pack Q/K/V destination-major with the existing Triton relayout kernel (reading through strides, no contiguous pre-pass), exchange in ONE collective, unpack with views + one contiguous pass. Bit-exact fallback to the unpacked path for ineligible inputs (CPU, GQA-mismatched shapes, non-contiguous, fp32, torch.compile). - _a2a_staging_buffer: reuse USP collective staging buffers keyed by (role, shape, dtype, device); bypassed under autograd, torch.compile and CUDA graph capture. - pack_qkv_destination_major gains an optional out= so the pack kernel writes straight into the reusable staging buffer. Pure data movement, bit-exact by construction and by test: 4-rank NCCL micro-benchmark asserts torch.equal across variants, and same-seed Wan2.2-T2V-A14B 4xH200 ulysses=4 frames are byte-identical to main. Perf (4xH200 NVLink, bf16): input exchange 2.950 -> 2.253 ms (720p), 1.331 -> 1.033 ms (480p), 2.112 -> 1.654 ms (H3 768p), 1.28-1.31x; pack kernel vs permute reference 2.7x. Wan2.2-A14B 480p x81f x20 steps, ulysses=4, 9 sequential requests drop-first median: e2e 32.653 -> 31.965 s per request (-2.1%), denoise step median 1535.3 -> 1483.3 ms (-3.4%) on an idle machine; an independent steady-contention sandwich pairing (base/opt/base) reproduces -1.7% e2e / -2.3% step. Adapted from NVlabs Sana sol-engine (Apache-2.0) MiniMax-H3 / Wan2.2 recipes; the relayout kernels were already ported in sgl-project#33275. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
saturn-acc
pushed a commit
to saturn-acc/sglang
that referenced
this pull request
Aug 16, 2026
Co-authored-by: zhenaozhenfu <zhenaozhenfu@minimaxi.com> Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: andyluo7 <andy.luo@amd.com> Co-authored-by: Zijie Xia <zijie_xia@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: chao-xue <877184285@qq.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Atituiset
pushed a commit
to Atituiset/sglang
that referenced
this pull request
Sep 10, 2026
Co-authored-by: zhenaozhenfu <zhenaozhenfu@minimaxi.com> Co-authored-by: BBuf <1182563586@qq.com> Co-authored-by: andyluo7 <andy.luo@amd.com> Co-authored-by: Zijie Xia <zijie_xia@icloud.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: chao-xue <877184285@qq.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This was referenced Sep 21, 2026
This was referenced Oct 1, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Scope
CI States
Latest PR Test (Base): 🚫 Run #30748870644
Latest PR Test (Extra): ❌ Run #30748870571