[ch-wan]

This PR is part of a stack (oldest at bottom):

• [Refactor] Declare every model's stages from a shared function and drop the meta-device neighbour build #43460
• -> [Refactor] Declare pipeline neighbours from a shared declaration function #43459
• [Refactor] Decide an exit's fixed facts when the stage is built #43458
• [Refactor] Bind a layer stack in one pass over each stage's data flow #43457
• [Refactor] Decide input-scattered eligibility and stage rejections in one place #43456
• [Refactor] Make residual read and update capabilities explicit and checked #43455
• [Fix] Keep residual rows and sums consistent across appends and pipeline handoffs #43454
• [Refactor] Record what a bound stage path does instead of reading it back #43453

Motivation

To bind the first and last stages of a pipeline rank's stack, make_layers built the neighbouring layers again on the meta device and read the stages they declared. Building a layer can have effects the meta device does not stop (streams, communicators, explicit devices), its choices had to be re-derived from modules nobody runs, and every bound stage kept its neighbour's and its producer's declaration, holding their reads, updates and transforms (and so their modules) for the life of the model.

Modifications

• Record a neighbouring stage by the facts binding reads of it. facts_of() reduces a declaration to the capabilities binding reads (UpdateFacts, ReadFacts, TransformFacts) without the declarations it is chained to; neighbours and recorded producers are such facts. A read's kernels are not among them: binding takes kernels only from the reads of the stages it binds, so ReadFacts supplies none. A rank whose first stage would have to run the output transform of an attention on the previous rank is rejected (no model declares one); a test covers it.
• Let a model declare its neighbouring stages without building the layers. make_layers / make_pp_layers take stage_facts: the model's shared declaration function, which returns the stages the layer at a global index declares, and which the layer also uses for its own declarations. Under PP each layer a rank builds is checked against it; a mismatch names the layer, the stage and each differing field with both values. Models not yet migrated stay on a transitional list that keeps the meta-device path; Nemotron-H, whose declarations already came from its pattern, gives a function and leaves the list. [Refactor] Declare every model's stages from a shared function and drop the meta-device neighbour build #43460 migrates every other model and removes the list.
• Drop stage boundary forwarding and exports nothing uses; drop the keyword arguments no output move is given.
• Describe neighbouring stages, exit policies and sum completers as they are in the layer boundary developer guide, including a table of exit policies and which of their decisions are fixed at construction.

Accuracy Tests

How the stack is checked:

• Unit tests: the 39 layer-boundary test files pass at the head of every PR in the stack, apart from the two AMD-only test_aiter_allreduce_fusion_amd cases that fail on NVIDIA on main too; the test files that changed most recently pass on every commit. The full test/registered/unit suite (926 files, one process per file) shows no failure on the stack's head that main does not have.
• Plan comparison: an out-of-tree harness binds every stage plan for seven stack shapes (decoder layers with and without an output transform, MHC, Nemotron-H patterns, Kimi K3, Qwen4-exp, NemotronNAS-style attention-only layers, single-stage appends) over 23 parallel configurations, four boundary reductions and one or two pipeline ranks (3,928 scenarios), and compares each bound plan leaf by leaf with the parent commit's.
• GPU: dummy weights are seeded per TP rank and scaled (0.05, norms 1.0), so that a sum taken twice or a mixture of rank partials changes the output; the default dummy weights give every rank the same shard and would hide both.
• GPU results below were measured on these commits as based on main 516fd1c1b5 unless marked as on the merge candidate; rebasing onto current main had no conflicts, and the CPU checks were run again after it.
• On the merge candidate (head of this stack against main 961bcf481d, H200, asymmetric dummy weights unless noted): DeepSeek-V2-Lite input-scattered PP2 is bit-identical to PP1 (main crashes on it); Nemotron-H cut between a mixer and its FFN differs from PP1 only at the rounding level of the FFN-to-mixer cuts (main: 0.93 and 2.21); Llama, Qwen3-MoE, GPT-OSS and DeepSeek-V2-Lite are bit-identical to main at PP1 and PP2; GSM8K (real DeepSeek-V2-Lite weights, 250 questions, TP4 with attention DP2 and the dense FFN fully data-parallel) gives 0.360 at PP2, 0.364 at PP1 and 0.368 on main at PP1, medians of three runs whose run-to-run spread is 0.012–0.016. The last revision of the stack changed only comments, docstrings, an equivalent condition and explicitly set attributes after these runs; its bound plans are identical.

For this PR: the plan comparison differs only in how a pipeline rank's neighbour appears in a bound plan (a ReadFacts instead of the read object), on an edge whose read does not run on the sending rank.

Speed Tests and Profiling

No forward-path change.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): 🚫 Run #38080873880
Latest PR Test (Extra): 🚫 Run #38080873610
Latest PR Test (AMD ROCm 10): 🚫 Run #38080873786

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🔴 Failed	–	Oct 10, 2026, 7:43 PM

💡 Tip: Enable Automations to automatically generate PRs for you.