Skip to content

[kimi k3][pd disagg] support pp prefill + dcp decode with dspark - #40045

Merged
Qiaolin-Yu merged 16 commits into
mainfrom
qiaolin/dspark-pd-prefill-pp
Sep 20, 2026
Merged

[kimi k3][pd disagg] support pp prefill + dcp decode with dspark#40045
Qiaolin-Yu merged 16 commits into
mainfrom
qiaolin/dspark-pd-prefill-pp

Conversation

@Qiaolin-Yu

@Qiaolin-Yu Qiaolin-Yu commented Sep 18, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Collaborator

Motivation

Input length / Concurrency TP8 PP1 input tokens/s TP4 PP2 input tokens/s Throughput change
8k / C16 9,667 11,654 +20.6%
8k / C64 9,658 11,556 +19.6%

Modifications

Accuracy Tests

aime26 verified

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): 🚫 Run #35496307454
Latest PR Test (Extra): ❌ Run #35496307325
Latest PR Test (AMD ROCm 10): 🚫 Run #35496307450

@Qiaolin-Yu Qiaolin-Yu changed the title [kimi k3][pd disagg] support prefill pp + dcp decode with dspark [kimi k3][pd disagg] support pp prefill + dcp decode with dspark Sep 18, 2026
YAMY1234
YAMY1234 reviewed Sep 19, 2026
View reviewed changes

@YAMY1234 YAMY1234 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went through the three parts (K3/kimi_linear PP-aware aux capture, non-last-rank DSparkWorkerV2, draft-head slicing in send_kvcache_dcp). The logic looks consistent for prefill PP + decode DCP, and the deferred boundary-layer capture in K3 checks out. A few comments inline.

Could you also fill in Motivation / Modifications and add the eval numbers (prefill-PP + decode-DCP vs. PP1)? Since this touches the KV transfer path it'd be good to have them on record.

Comment thread python/sglang/srt/model_executor/model_runner.py Outdated
Comment thread python/sglang/srt/disaggregation/mooncake/conn.py
Comment thread python/sglang/srt/speculative/dspark_components/dspark_worker_v2.py
@Qiaolin-Yu Qiaolin-Yu added run-ci CI: run the baseline test suite on this PR bypass-fastfail labels Sep 20, 2026
@Qiaolin-Yu
Qiaolin-Yu requested a review from YAMY1234 September 20, 2026 01:11
kpham-sgl
kpham-sgl approved these changes Sep 20, 2026
View reviewed changes

@kpham-sgl kpham-sgl left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, very clean

@kpham-sgl kpham-sgl added the release-highlight Candidate PR for release note highlight label Sep 20, 2026
@Qiaolin-Yu

Qiaolin-Yu commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/managers/test_disagg_idle_step_counters.py

@github-actions

github-actions Bot commented Sep 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/unit/managers/test_disagg_idle_step_counters.py:

🚀 ubuntu-latest (1 test): ✅ View workflow run 

cd test/ && python3 registered/unit/managers/test_disagg_idle_step_counters.py

@Qiaolin-Yu

Qiaolin-Yu commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

https://github.com/sgl-project/sglang/actions/runs/35487510501/job/106016608715

@Qiaolin-Yu

Qiaolin-Yu commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/managers/test_disagg_idle_step_counters.py

@github-actions

github-actions Bot commented Sep 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/unit/managers/test_disagg_idle_step_counters.py:

🚀 ubuntu-latest (1 test): ❌ View workflow run 

cd test/ && python3 registered/unit/managers/test_disagg_idle_step_counters.py

@Qiaolin-Yu

Qiaolin-Yu commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/managers/test_disagg_idle_step_counters.py

@github-actions

github-actions Bot commented Sep 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/unit/managers/test_disagg_idle_step_counters.py:

🚀 ubuntu-latest (1 test): ✅ View workflow run 

cd test/ && python3 registered/unit/managers/test_disagg_idle_step_counters.py

@Qiaolin-Yu
Qiaolin-Yu merged commit f4c2563 into main Sep 20, 2026
69 of 169 checks passed
@Qiaolin-Yu
Qiaolin-Yu deleted the qiaolin/dspark-pd-prefill-pp branch September 20, 2026 07:15
@kpham-sgl kpham-sgl mentioned this pull request Sep 20, 2026
Open
19 of 39 tasks
@zhangxiaolei123456 zhangxiaolei123456 mentioned this pull request Sep 20, 2026
Closed
5 tasks
@kpham-sgl kpham-sgl mentioned this pull request Sep 20, 2026
Merged
This was referenced Sep 20, 2026
Merged
Closed
This was referenced Sep 21, 2026
Open
Open
This was referenced Sep 24, 2026
Open
Closed
This was referenced Sep 29, 2026
Closed
Open
This was referenced Oct 11, 2026
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@YAMY1234 YAMY1234 YAMY1234 approved these changes
@kpham-sgl kpham-sgl kpham-sgl approved these changes
@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner
@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner
@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner
@ShangmingCai ShangmingCai Awaiting requested review from ShangmingCai ShangmingCai is a code owner
@ByronHsu ByronHsu Awaiting requested review from ByronHsu ByronHsu is a code owner
@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw is a code owner
@Duyi-Wang Duyi-Wang Awaiting requested review from Duyi-Wang Duyi-Wang is a code owner
@sogalin sogalin Awaiting requested review from sogalin sogalin is a code owner
@Fridge003 Fridge003 Awaiting requested review from Fridge003 Fridge003 is a code owner
@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

Assignees

No one assigned

Labels

bypass-fastfail release-highlight Candidate PR for release note highlight run-ci CI: run the baseline test suite on this PR speculative-decoding

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Qiaolin-Yu @YAMY1234 @kpham-sgl