Repository navigation
[minimax-m3] Split 1/4: sparse attention ops + JIT kernels + config foundation - #28712
Merged
Fridge003 merged 10 commits intoJun 22, 2026
Merged
Conversation
…oundation First of a 4-PR split of sgl-project#27944 (MiniMax-M3). This PR is purely additive on top of main: new compute kernels, their tests/benchmarks, and the config helpers/env descriptors they depend on. It does not touch any generic forward / quant / attention runtime path, so it is dead-on-main and green without an e2e run (e2e is deferred to PR4, the only PR that moves forward/quant/attention behavior). Contents: - minimax_sparse_ops/** (decode/prefill/naive sparse attention kernels + tests) - jit_kernel minimax/m3/moe_topk_sigmoid kernels (C++/CUDA headers + Python wrappers + tests/benchmarks), including store_kv_index (depended on by PR2) - guarded AMD gfx95 MXFP8 MoE kernels + triton config JSONs - additive MoE-runner changes: interleaved dataclass default (True), MXFP8 branch in triton runner (guarded by use_mxfp8=False default) - model_config: is_minimax_sparse + get_minimax_sparse_attention_config and helpers + mxfp8 dtype registration + piecewise VL guard - environ: 3 MiniMax descriptors needed by the kernels above (SGLANG_OPT_USE_MINIMAX_DECODE_TOPK_RADIX, SGLANG_MINIMAX_M3_FUSED_SWIGLU_MXFP8, SGLANG_MINIMAX_M3_FUSED_MOE_COMBINE) - docker/rocm.Dockerfile: triton_kernels build step for gfx950 Excluded (moved to PR2/PR3/PR4 by the split design): - mem-cache / HiCache / sparse pool + their env descriptors (PR2) - disagg (PR3) - model + VL + glue + function-call, the fp8/fp8_kernel/fp8_utils/entrypoint quant group (default-path changes), deep_gemm/radix_attention (generic hunk changes + cross-bucket imports), and generic MoE infra RED files (PR4)
JustinTong0323
requested review from
AniZpZ,
BBuf,
DarkSharpness,
Edwardf0t1,
FlamingoPg,
Fridge003,
HaiShaw,
HydraQYH,
Qiaolin-Yu,
Ying1123,
b8zhong,
celve,
ch-wan,
hebiao064,
ishandhanani,
ispobock,
merrymercy,
sogalin,
yctseng0211 and
yuan-luo
as code owners
June 19, 2026 06:29
github-actions
Bot
added
quant
This was referenced Jun 19, 2026
This test exercises per_token_quant_fp8_ue8m0_scatter (PR1's minimax_quant_ue8m0) against fill_gateup_input_triton_kernel, whose signature source-PR extended to 13 params (+m_max, scale_row_stride, scale_col_stride, SCALE_MN_MAJOR). That signature change lives in ep_moe/kernels.py, which is in split PR4 (generic MoE infra RED bucket). On main the kernel is still the 10-param form, so this test fails here with 'dynamic_func() got multiple values for argument BLOCK_SIZE'. Move the test to PR4 so it lands together with the kernel signature change it depends on. PR4 rebases onto a main that already has PR1, so the test can import minimax_quant_ue8m0 from PR1 and the 13-param kernel from PR4.
This test asserts equivalence of the fused qkv+index mxfp8 deep_gemm matmul -- it only exercises _deepgemm_w8a8_mxfp8_linear_with_fallback and the scale packing in fp8_utils (both in split PR4's fp8 quant group). It imports nothing from PR1's minimax jit_kernel. On main fp8_utils lacks that function, so it fails here with ImportError. Move to PR4 to land with the fp8 quant group.
The MXFP8->block-fp8 [1,32]->[128,128] remap in the fused_moe_triton tuning helper was unconditional on architecture, but the weight conversion it mirrors only happens on gfx942 (MI300X, no MX matmul HW). NVIDIA (sm100) and gfx95 run MXFP8 natively at [1,32], so tuning remapped them to [128,128] and the resulting config filename (block_shape=[128,128]) never matched what serving queries ([1,32]) on those platforms — the tuned config silently went unused. Gate the remap on is_hip() and not is_gfx95_supported(), mirroring fp8.py's native-MXFP8 check, so only gfx942 remaps.
… changes Compress the 3 environ descriptors added in this split (3->1 line for the radix-select one) and the gfx942 gate comment in common_utils (6->2 lines). Comment-only; no behavior change.
Fridge003
reviewed
Jun 21, 2026
Fridge003
reviewed
Jun 21, 2026
… interleaved) - triton.py: gate the mxfp8_amd_gfx95 import on is_hip() and is_gfx95_supported() in both run paths — the kernel is gfx95-only, so use_mxfp8 alone must not route non-gfx95 (gfx942/NVIDIA) through it. - base.py / fused_moe.py / layer.py: rename the MoE 'interleaved' flag to gate_up_interleaved with a comment clarifying it only affects the silu+is_gated swiglu path (interleaved gate/up weight layout vs split).
…il-closed - Finish the gate_up_interleaved rename in triton.py (call sites) and mxfp8_moe_amd_gfx95.py (fused_experts_mxfp8 param + use) missed in the prior commit. - Add a fail-closed guard: Triton MoE runner does not support NVIDIA MXFP8 (only gfx95 has the native MXFP8 Triton kernel). On CUDA, raise instead of silently falling through to the non-mxfp8 fp8 path (which would mis-handle mxfp8 weights). Mirrors the gfx95 gate; the proper NVIDIA MXFP8 path is DeepGemm (auto-selected via server_args M3 SM100 default, lands in split PR4).
Fridge003
approved these changes
Jun 22, 2026
JustinTong0323
added
the
run-ci-extra
Collaborator
Author
|
/rerun-failed-ci |
JustinTong0323
pushed a commit
to JustinTong0323/sglang
that referenced
this pull request
Jul 1, 2026
… + generic infra Rebased onto main after sgl-project#28712/sgl-project#28713/sgl-project#28714 landed. - test_per_token_group_quant_8bit.py: kept both tests (sgl-project#29636 shape coverage + PR4 JIT-vs-AOT v2 byte-identical) - multi_tokenizer_mixin.py: resolved to main version (sgl-project#28573 removed SGLANG_GRANIAN_PARENT_PID)
Chronostasys
pushed a commit
to MindLab-Research/sglang
that referenced
this pull request
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 1 of a 4-PR split of #27944 (MiniMax-M3). This PR is purely additive on top of
main: new compute kernels, their tests/benchmarks, and the config helpers / env descriptors they depend on. It does not touch any generic forward / quant / attention runtime path, so it is dead-on-main and green without an e2e run.What's in this PR
minimax_sparse_ops/**— decode / prefill / naive sparse attention kernels + unit tests (self-contained, no model/hf_config import)jit_kernelMiniMax / M3 /moe_topk_sigmoidkernels — C++/CUDA headers + Python wrappers + tests/benchmarks, includingstore_kv_index(depended on by split PR2)interleaveddataclass default (True), MXFP8 branch in the triton runner (guarded byuse_mxfp8=Falsedefault)model_config:is_minimax_sparse+get_minimax_sparse_attention_configand helpers +mxfp8dtype registration + piecewise VL guardenviron: 3 MiniMax descriptors needed by the kernels above (SGLANG_OPT_USE_MINIMAX_DECODE_TOPK_RADIX,SGLANG_MINIMAX_M3_FUSED_SWIGLU_MXFP8,SGLANG_MINIMAX_M3_FUSED_MOE_COMBINE)docker/rocm.Dockerfile:triton_kernelsbuild step for gfx950Why it's green-on-main
Every modified shared file was audited hunk-by-hunk. Changes are limited to:
elif is_deep_gemm()/is_triton()/_is_hip and _is_gfx95_supported) whose default on main is unchanged,interleaved=True) that preserve existing call behavior,No generic forward / quant / attention default path is altered.
Split plan (for reviewers)
PR4 is the only PR that moves forward / quant / attention behavior and will carry the devbox e2e (GSM8K stop-rate + acc). The default-path-changing quant files (
fp8.py/fp8_kernel.py/fp8_utils.py/entrypoint.py) and the cross-bucket-importingdeep_gemm.py/radix_attention.pyare intentionally kept for PR4.CI States
Latest PR Test (Base): ⏳ Run #27980472842
Latest PR Test (Extra): ⏳ Run #27980472345