[DeepSeek V4] Enable FlashMLA sparse prefill by default by YAMY1234 · Pull Request #29775 · sgl-project/sglang

YAMY1234 · 2026-06-30T16:55:04Z

Motivation

FlashMLA sparse prefill improves DeepSeek V4 server-side prefill throughput across the tested input lengths under saturated serving load. Enable the existing path by default while preserving the existing size-based selection policy when the flag is explicitly disabled.

Modifications

Change the default value of SGLANG_OPT_FLASHMLA_SPARSE_PREFILL from false to true. Setting it to 0 restores the existing size-based selector; queries above the existing threshold continue to select sparse prefill as before.

Accuracy Tests

No kernel or model-forward implementation is changed. Verified that the flag resolves to true when unset and still resolves to false when explicitly disabled.

Speed Tests and Profiling

Measured on GB300 with TP4 / DP4 / EP4 prefill, DP attention, FP8 KV cache, an 8,192-token prefill chunk per rank, and persistent request backlog. Each 1K–16K arm uses a fixed 300-second measurement window with zero failed requests and zero cached tokens.

Nominal ISL	Global concurrency	OFF TPS/rank	ON TPS/rank	Gain	Replay gain
1K	128	15,818.18	16,485.28	+4.217%	+4.242%
8K	64	13,870.58	14,813.02	+6.795%	+6.883%
16K	64	13,464.29	14,545.99	+8.034%	+7.914%
128K	8	10,493	11,792	+12.4%	—

The maximum difference between the primary and replay gains for the 1K–16K measurements is 0.12 percentage points. The performance claim is scoped to the validated GB300 serving configurations shown above. This remains a draft while underfilled batches, SM120, and prefill context parallelism are outside the current safety-validation scope.

Checklist

pre-commit run --files python/sglang/srt/environ.py
Verified default-on and explicit-false flag behavior.
No documentation update is required; the existing flag remains unchanged.

CI States

Latest PR Test (Base): ❌ Run #28544931569
Latest PR Test (Extra): ⏳ Run #28546902349

gemini-code-assist · 2026-06-30T16:55:08Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

gemini-code-assist · 2026-06-30T17:08:06Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

Fridge003 · 2026-06-30T22:11:59Z

/rerun-test test/registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_megamoe_b200.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_h200.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_h200.py

github-actions · 2026-06-30T22:12:29Z

Results for /rerun-test test/registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_megamoe_b200.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_h200.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_h200.py:

🚀 4-gpu-b200 (2 tests): ❌ View workflow run

cd test/ && python3 registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py
cd test/ && python3 registered/models_e2e/test_deepseek_v4_flash_fp4_megamoe_b200.py

🚀 8-gpu-h200 (1 test): ✅ View workflow run

cd test/ && python3 registered/models_e2e/test_deepseek_v4_flash_fp4_h200.py

YAMY1234 · 2026-07-01T01:54:21Z

/rerun-test test/registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py

github-actions · 2026-07-01T01:54:46Z

Results for /rerun-test test/registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py:

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py

Fridge003 · 2026-07-01T09:39:33Z

/rerun-test test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py

github-actions · 2026-07-01T09:39:55Z

Results for /rerun-test test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py:

🚀 4-gpu-b200 (1 test): ❌ View workflow run

cd test/ && python3 registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py

Fridge003 · 2026-07-01T20:15:51Z

/rerun-test test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py

github-actions · 2026-07-01T20:16:24Z

Results for /rerun-test test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py:

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py

…29775) Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>

…le. reference from pull sgl-project#29775 Change-Id: Ic6f2e424b791591260e9a1977b94f313ba25384a

perf(dsv4): enable FlashMLA sparse prefill by default

8bc9eef

YAMY1234 marked this pull request as ready for review June 30, 2026 17:08

fix(dsv4): share sparse prefill workspace across graph buckets

5a1d5c4

YAMY1234 requested review from Fridge003, HaiShaw, Qiaolin-Yu, hebiao064, ispobock and merrymercy as code owners July 1, 2026 01:54

github-actions Bot added the deepseek label Jul 1, 2026

Disable sparse prefill for DeepSeek V4 CP

5b80388

Fridge003 approved these changes Jul 1, 2026

View reviewed changes

Fridge003 merged commit c865347 into sgl-project:main Jul 1, 2026
90 of 104 checks passed

Fridge003 added the release-highlight Candidate PR for release note highlight label Jul 1, 2026

Oasis-Git mentioned this pull request Jul 2, 2026

Revert "[DeepSeek V4] Enable FlashMLA sparse prefill by default" #29880

Closed

YAMY1234 mentioned this pull request Jul 2, 2026

[DeepSeek V4] Cover both dense and sparse prefill paths in the compress attention unittest #29885

Merged

4 of 5 tasks

vstone-w pushed a commit to ClownBin/sglang that referenced this pull request Jul 2, 2026

[DeepSeek V4] Enable FlashMLA sparse prefill by default (sgl-project#…

e98a6d1

…29775) Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>

michaelzhang-ai mentioned this pull request Jul 2, 2026

[AMD][DeepSeek V4] Fix default FlashMLA sparse prefill off on ROCm/HIP #29982

Merged

1 of 2 tasks

This was referenced Jul 2, 2026

[CI Monitor] Daily Report - 2026-07-02 bingxche/sglang-ci-bot#122

Open

[CI Monitor] Daily Report - 2026-07-03 bingxche/sglang-ci-bot#123

Open

This was referenced Jul 4, 2026

[CI Monitor] Daily Report - 2026-07-04 bingxche/sglang-ci-bot#125

Open

[CI Monitor] Daily Report - 2026-07-05 bingxche/sglang-ci-bot#126

Open

[CI Monitor] Daily Report - 2026-07-06 bingxche/sglang-ci-bot#127

Open

amd-bot mentioned this pull request Jul 7, 2026

[CI Monitor] Daily Report - 2026-07-07 bingxche/sglang-ci-bot#128

Open

amd-bot mentioned this pull request Jul 15, 2026

[CI Monitor] Daily Report - 2026-07-15 bingxche/sglang-ci-bot#136

Open

ischindl mentioned this pull request Jul 17, 2026

[Feature] Native SM120 (Blackwell) support for flash_mla_sparse_fwd (DeepSeek-V4 sparse-attention prefill) #31578

Closed

alisonshao mentioned this pull request Jul 19, 2026

[DeepSeek-V4] Fix idle-rank dummy-extend sparse-prefill crash under DP breakable CUDA graph #31705

Merged

Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026

[DeepSeek V4] Enable FlashMLA sparse prefill by default (sgl-project#…

a09da7d

…29775) Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>

YAMY1234 deleted the yamy1234/dsv4-flashmla-sparse-prefill-default-on branch August 25, 2026 16:17

yangxin-mx pushed a commit to MetaX-MACA/sglang that referenced this pull request Oct 10, 2026

fix: MC3-14290 close SGLANG_OPT_FLASHMLA_SPARSE_PREFILL when pcp enab…

664d145

…le. reference from pull sgl-project#29775 Change-Id: Ic6f2e424b791591260e9a1977b94f313ba25384a

Navigation Menu

Repository navigation

[DeepSeek V4] Enable FlashMLA sparse prefill by default - #29775

[DeepSeek V4] Enable FlashMLA sparse prefill by default#29775
Fridge003 merged 3 commits into
sgl-project:mainfrom
YAMY1234:yamy1234/dsv4-flashmla-sparse-prefill-default-on

YAMY1234 commented Jun 30, 2026 •

edited by github-actions Bot

Loading

Uh oh!

gemini-code-assist Bot commented Jun 30, 2026

Uh oh!

gemini-code-assist Bot commented Jun 30, 2026

Uh oh!

Fridge003 commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

YAMY1234 commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026 •

edited

Loading

Uh oh!

Fridge003 commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026 •

edited

Loading

Uh oh!

Fridge003 commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

YAMY1234 commented Jun 30, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

CI States

Uh oh!

gemini-code-assist Bot commented Jun 30, 2026

Uh oh!

gemini-code-assist Bot commented Jun 30, 2026

Uh oh!

Fridge003 commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

YAMY1234 commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Fridge003 commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Fridge003 commented Jul 1, 2026

Uh oh!

github-actions Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

YAMY1234 commented Jun 30, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Jun 30, 2026 •

edited

Loading

github-actions Bot commented Jul 1, 2026 •

edited

Loading

github-actions Bot commented Jul 1, 2026 •

edited

Loading

github-actions Bot commented Jul 1, 2026 •

edited

Loading