Repository navigation
[Feature] Kimi K3 Roadmap #32607
Description
Links
- Cookbook: https://docs.sglang.io/cookbook/autoregressive/Moonshotai/Kimi-K3
- Day0 PR: [Kimi] Support kimi-k3 #32541
- Blog: https://www.lmsys.org/blog/2026-07-27-kimi-k3-day0-support
- DSpark: https://huggingface.co/RadixArk/Kimi-K3-DSpark
- Bug Tracking: Kimi-K3 bug tracking #32970
Roadmap
Kernel Optimization
- Continue optimize performance in bs=1/64 and TP8(GB300)
- Integrate custom KDA Prefill kernel and make it compatible with RadixCache
- Refactor to generalize kernel supporting different parallelism.
- Support GEMM Allreduce fusion
- KDA MTP decode
- BF16 gemm / gemm fusion with SiTU
- Optimize kernels in TP16xhopper
Parallelism
-
Decode Context Parallelism (DCP) Enhancement. See [Roadmap][DCP] Decode Context Parallelism & Helix Parallelism (2026 Q4) #29736
-
Chunked Pipeline Parallelism Enhancement
- Support PP + Spec + PD Disagg
-
Integrate MoonEP for better all-to-all performance https://github.com/sgl-project/sglang/tree/moonep
Extending Unified Memory Compatibility
- Attention backends
- Speculative Decoding
- PD disaggregation
- Hi-cache
DSpark
- Enhance its capability on longer context https://huggingface.co/RadixArk/Kimi-K3-DSpark
Code Cleaning
- Merge Day0 PR [Kimi] Support kimi-k3 #32541
- Merge DeepGemm patch into [sglang fork][(https://github.com/sgl-project/DeepGEMM/tree/dev) and upgrade sgl-deep-gemm wheel https://github.com/[SM100] Add Kimi-K3 SiTU activation DeepGEMM#67
- Upgrade Flashinfer after the cutedsl+dcp patch (feat(mla): support packed low-head and variable-Q decode flashinfer-ai/flashinfer#4178) and trtllm moe patch ([feat] Add SITU trtllmgen MOE flashinfer-ai/flashinfer#4180) has been included Bump FlashInfer to 0.6.17 and remove Kimi K3 workarounds #33997
- Build a custom wheel for DeepEP, and include the patch in https://github.com/sgl-project/sglang/blob/bd51cab0c01075c9b73cc1dfae9d27b7b0c95619/docker/kimi_k3/apply_deepep_k3_patch.sh
Slack
Join [#kimi-k3] channel at https://slack.sglang.io/
Activity
foraxe commentedon Jul 29, 2026
Maybe we could also explore some of these approaches:
foraxe commentedon Jul 30, 2026
kpham-sgl commentedon Jul 31, 2026
Hi @foraxe, we can keep the discussions under #29736
foraxe commentedon Jul 31, 2026
Got it. Thank you.
wirybeaver commentedon Aug 2, 2026
BF16 MoonEP PoC handoff is ready for the Kimi-K3 roadmap item.
NV18) passed the BF16 dispatch -> prefetch -> compute -> combine correctness harness on 8 ranks withglobal_ok=true,max_abs_err=0.0,relative_err=0.0.nvpohanh commentedon Aug 5, 2026
Will #32281 cover this?
charliechenye commentedon Aug 12, 2026
Quick design question coming out of reviewing #33249: what do we want the MoonEP + batch-overlap (BTO) contract to be in SGLang?
The current PoC routes MoonEP through MaybeTboDeepEPDispatcher. When TBO is enabled, that wrapper creates two MoonEPDispatcher instances. At the same time, both dispatchers appear to resolve the process wide MoonEPBuffer using the same static key (shape/topology + process group), so they would reuse the same underlying moonep.Buffer.
I may be missing some intended MoonEP semantics here, so wanted to check with the team before treating this as a bug:
My instinct for the current PoC would be to keep the supported surface narrow: disable overlap first, then add it once the buffer/plan ownership model is explicit. But I would love to understand the intended direction.
liuqi-1 commentedon Aug 13, 2026
I only implemented PP+DSpark. It needs some adaptions to support PD.