Skip to content

[Fix] [AMD] GLM5: Fix MTP layer_quant_config in-place mutation and add nextn Quark-exclude unit test - #30565

Open
Raiden-Makoto wants to merge 10 commits into
sgl-project:mainfrom
Raiden-Makoto:tmp/eagle-mtp
Open

Raiden-Makoto wants to merge 10 commits into
sgl-project:mainfrom
Raiden-Makoto:tmp/eagle-mtp

Conversation

@Raiden-Makoto

@Raiden-Makoto Raiden-Makoto commented Jul 8, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Follow-up to #30265 / #29781 (supersedes #30379). #30265 merged with two gaps flagged in its own review:

  1. The GLM-5.2 Quark-MXFP4 MTP path has no PR-CI coverage — the only AMD GLM-MXFP4 test is nightly=True and does not enable MTP; the GLM MTP e2e test is CUDA-only + FP8, not Quark MXFP4.
  2. GlmMoeDsaForCausalLMNextN._resolve_nextn_quant_config remaps layer_quant_config keys by mutating the shared quant_config in place, which can corrupt the main model's per-layer scheme selection (gemini-code-assist review on [AMD] Fix GLM-5.2 MTP Quark excludes #30265).

The fix and the CPU test here originate from #30379 (@fanxingran, co-authored).

Modifications

  • python/sglang/srt/models/glm4_moe.py: _resolve_nextn_quant_config copy.copys quant_config (and its nested dict) before any remap, and only copies when a remap/exclude is actually needed — no shared-state mutation.
  • python/sglang/srt/models/deepseek_nextn.py: docstring on the base _resolve_nextn_quant_config documenting the subclass override contract.
  • test/registered/unit/models/test_nextn_quark_exclude.py (new, CPU CI base-a-test-cpu): base DeepseekV3ForCausalLMNextN passthrough (non-quark / None), GLM-5.2 MTP exclude_layers + layer_quant_config remap, coarse fused-expert prefix, and the no-mutation guarantee.
  • test/registered/amd/accuracy/mi35x/test_glm52_mxfp4_mtp_eval_mi35x.py (new, nightly nightly-amd-8-gpu-mi35x-glm52-mxfp4-mtp): 8-GPU GLM-5.2-MXFP4 EAGLE-MTP GSM8K eval — GPU coverage that the bf16 nextn layer loads under Quark MXFP4 and MTP accepts drafts. Wired into nightly-test-amd.yml and nightly-test-amd-rocm720.yml.

Accuracy Tests

CPU unit test test/registered/unit/models/test_nextn_quark_exclude.py on this branch:

6 passed, 5 warnings in 10.64s

test_layer_quant_config_keys_remapped_without_mutation fails on current main (exercises the #30265 in-place mutation); green after this change. pre-commit run passes all hooks (isort/ruff/black/yaml/duplicate-job-name/registry validation).

Nightly GPU eval thresholds are the reported amd/GLM-5.2-MXFP4 MI355x numbers: GSM8K accuracy floor 0.935 (#29781 measured 0.950 at EAGLE 5/1/6, the fixture's config) and accept-length floor 5.0 (#30265 measured 5.94).

Speed Benchmarks

N/A — quant-config resolution runs once at model construction; no inference-path change.

Checklist


CI States

Latest PR Test (Base): 🚫 Run #33544291185
Latest PR Test (Extra): ❌ Run #33544290924
Latest PR Test (AMD ROCm 7.2): ❌ Run #33544291139

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

…de tests

GlmMoeDsaForCausalLMNextN._resolve_nextn_quant_config (sgl-project#30265) remapped
layer_quant_config keys by mutating the shared quant_config in place, which can
corrupt the main model's per-layer scheme selection (flagged in the sgl-project#30265
review). Copy quant_config and its nested dict before any mutation, and only
copy when a remap/exclude is actually needed.

Tests (the missing sgl-project#30265 coverage):
- test/registered/unit/models/test_nextn_quark_exclude.py (CPU CI,
  base-a-test-cpu): base DeepseekV3ForCausalLMNextN passthrough plus the
  GLM-5.2 MTP exclude / layer_quant_config remap and no-mutation guarantees.
- test/registered/amd/accuracy/mi35x/test_glm52_mxfp4_mtp_eval_mi35x.py
  (nightly-amd-8-gpu-mi35x-glm52-mxfp4-mtp): 8-GPU GLM-5.2-MXFP4 EAGLE-MTP
  GSM8K eval verifying the bf16 nextn layer loads under Quark MXFP4; wired into
  nightly-test-amd.yml and nightly-test-amd-rocm720.yml.

Fix + CPU test originally by @fanxingran (sgl-project#30379); references sgl-project#29781.

Co-authored-by: fanxingran <xingran.fan@amd.com>
@Raiden-Makoto
Raiden-Makoto marked this pull request as ready for review July 9, 2026 14:35
@HaiShaw

HaiShaw commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Aug 17, 2026
@Raiden-Makoto

Copy link
Copy Markdown
Contributor Author

@amd-bot ci-status

@Raiden-Makoto

Raiden-Makoto commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor Author

CI tests failed due to rate limit.

@Raiden-Makoto

Copy link
Copy Markdown
Contributor Author

/rerun-failed-ci

1 similar comment
@Raiden-Makoto

Copy link
Copy Markdown
Contributor Author

/rerun-failed-ci

Raiden-Makoto and others added 3 commits August 25, 2026 16:12
# Conflicts:
#	.github/workflows/nightly-test-amd-rocm720.yml
# Conflicts:
#	.github/workflows/nightly-test-amd-rocm720.yml
@Raiden-Makoto

Copy link
Copy Markdown
Contributor Author

/rerun-failed-ci

@Raiden-Makoto Raiden-Makoto changed the title [AMD] [GLM5] Fix MTP layer_quant_config in-place mutation + add nextn Quark-exclude unit test [Fix] [AMD] GLM5: Fix MTP layer_quant_config in-place mutation and add nextn Quark-exclude unit test Oct 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

amd deepseek run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants