[maocheng23]

Motivation

#42297 added --speculative-use-block-verification (Block Verification Accelerates Speculative Decoding, Algorithm 2) for linear EAGLE / EAGLE3 / NEXTN only. The DFlash family (DFLASH, DSPARK) already verifies sampled drafts through the same Triton chain sampler, but the flag was rejected for these algorithms at startup.

Modifications

• Startup gate (speculative_hook.py): accept DFLASH and DSPARK with --speculative-use-block-verification on CUDA/ROCm. NPU stays rejected because sgl_kernel_npu's chain sampler has no block mode. Only EAGLE/EAGLE3 still auto-enable --speculative-use-rejection-sampling. DFlash-family sampled paths always pass their proposal distribution and never read that flag.
• Shared accept path (dspark_accept.py): AcceptSampling / accept_sampling / _accept_sampling_core take an explicit block_verification keyword and call chain_speculative_sampling_triton(..., block_verification=True) when it is set. The kernel is unchanged from [Speculative] Add opt-in block verification #42297.
• Callers:
  • DSpark: DSparkWorkerV2 passes the flag to TargetVerifyExecutor, which forwards it through accept_draft_tokens to both the all-sampling branch and the mixed greedy/sampling branch.
  • DFlash: DFlashWorkerV2._selector_sampling_accept forwards it. This method serves the candidate-selector drafts and the sampled LiLiCorr drafts (SGLANG_ENABLE_LILICORR_SAMPLING=1).
• Greedy DFlash drafts: plain DFlash, Domino, and LiLiCorr without sampling propose argmax tokens. The proposal is a point mass, so q(x_i) = 1, r_i = prod p(x_j), and token-wise verification already reaches the per-prefix upper bound P(accept >= i) = r_i. Block verification therefore accepts the same prefixes, so these paths keep the existing target-only verifier. The worker logs a warning that the flag has no effect.
• Ragged/cutoff verify (cutoff_verify_lens) still truncates the sampler's result afterward, as it does in token-wise mode. A prefix of a target-distributed sample is still target-distributed, so the cutoff does not affect correctness.
• Updated the docs and the CLI help.

Accuracy Tests

• test_block_verification_args.py:
  • DFLASH/DSPARK are accepted, and block verification does not switch on rejection sampling for them. The per-algorithm handlers are patched out because they need a real draft checkpoint.
  • DFLASH/DSPARK on NPU are rejected.
  • The rejected-algorithm case moves from DFLASH to NGRAM.
• test_block_verification.py::test_dflash_family_accept_uses_block_verification (GPU): calls DSpark's real accept_draft_tokens, in both the all-sampling and the mixed-greedy branches. The proposal uses q(x1) = 1, p(x1) = 1/2 and q(x2) = 1/2, p(x2) = 1, so r_1 = 1/2 and r_2 = 1.
  • With block verification, every sampled row accepts both drafts and the bonus token from the target.
  • Without it, each row rejects the first draft with probability 1/2, so the test asserts that at least one of 63 rows does.

Local (macOS, CPU, Torch 2.14.1):

PYTHONPATH=python python -m pytest -q \
  test/registered/unit/spec/test_block_verification_args.py \
  test/registered/unit/spec/test_spec_registry.py \
  test/registered/unit/spec/test_eagle_gate_routing.py \
  test/registered/unit/spec/test_spec_cpu_overlap_constraint.py
# 45 passed, 21 subtests passed

GPU (one NVIDIA GB300; Torch 2.14.1+cu130, sglang-kernel 0.4.9, FlashInfer 0.7.0.post1, Triton 3.8.0, Transformers 5.17.0; source snapshot of c8699a8046):

python -m pytest -q \
  test/registered/kernels/ops/speculative/test_block_verification.py \
  test/registered/unit/spec/test_block_verification_args.py \
  test/registered/spec/dspark/test_dspark_kernel_parity.py
# 30 passed, 43 subtests passed

Pre-commit passed on all changed files.

Speed Tests and Profiling

Qwen3.8-27B + DFlash2 / AIME25 end-to-end

Block verification increased acceptance length on all three seed pairs, by +3.31% pooled. That is larger than the +0.88% that #42297 measured for MTP with three proposed tokens. DFlash2 proposes seven tokens per step, so there are more prefixes for block verification to rescue after an early rejection. The pooled throughput gain of +1.76% was not consistent: per seed it was +2.75%, +2.67%, and -0.29%. The accuracy and cap-hit differences are within run-to-run noise for three sampled runs.

Metric	Token-wise	Block	Change
Acceptance length (includes bonus)	4.0091	4.1416	+3.31%
Correct drafts per verify step	3.0091	3.1417	+4.40%
Output throughput (tokens/s)	1260.93	1283.17	+1.76%
Boxed score over entire response	76/90	76/90
Responses hitting the 32,768-token cap	18/90	21/90

Seed	Token AL	Block AL	Token tok/s	Block tok/s	Token correct	Block correct
980406	4.0106	4.1573	1219.29	1252.76	25/30	24/30
981406	3.9919	4.1371	1260.97	1294.64	24/30	27/30
982406	4.0256	4.1306	1306.96	1303.13	27/30	25/30

Setup:

• Hardware and models: one NVIDIA GB300, TP1. Target Qwen/Qwen3.8-27B BF16, revision 1d4bf0f2ff. Draft incoai/Qwen3.8-27B-DFlash2, revision 015e795645 (DFlash2DraftModel: candidate selector with top-k 16, block size 8).
• Speculative settings: --speculative-algorithm DFLASH --speculative-num-draft-tokens 8.
• Server settings: BF16 KV cache, FP32 recurrent state, FlashInfer attention, CUDA graphs on, radix cache off, concurrency 8.
• The two arms: they run identical commands, except that the block arm adds --speculative-use-block-verification. The server log confirms that selector sampling is folded into the draft CUDA graph (sampling_enabled=True), and neither arm logged the no-effect warning.

Protocol: same as #42297.

• Workload: all 30 MathArena/aime_2025 questions, with the same frozen dataset (SHA256 b4bdeba5...). Thinking is enabled at xhigh. Sampling uses temperature 1.0, top-p 0.95, and top-k 20, with at most 32,768 new tokens.
• Seeds and arm order: three base seeds, offset by problem ID. Arm order alternates by seed.
• Warmup: each run warms up eight slots and flushes the cache before timing.
• Metrics: acceptance length is total completion tokens divided by total verification steps. Throughput is total output tokens divided by measured wall time. An independent pass over the raw per-request records reproduced every aggregate and checked request coverage, the sampling seeds, and that the commands match.

Qwen3.8-27B + DSpark / AIME25 end-to-end

DSpark shows the same effect. Block verification increased acceptance length on all three seed pairs, by +3.13% pooled (+3.02%, +2.42%, +3.96% per seed). Throughput was flat at +0.54% pooled, and per seed it was +1.51%, +0.42%, and -0.45%. Other jobs were running on the same node's other GPUs during this run, so its throughput is noisier than the DFlash2 run's. The differences in score and cap hits are within run-to-run noise for three sampled runs.

Metric	Token-wise	Block	Change
Acceptance length (includes bonus)	4.0134	4.1390	+3.13%
Correct drafts per verify step	3.0135	3.1393	+4.17%
Output throughput (tokens/s)	1296.61	1303.58	+0.54%
Boxed score over entire response	77/90	82/90
Responses hitting the 32,768-token cap	19/90	17/90

Seed	Token AL	Block AL	Token tok/s	Block tok/s	Token correct	Block correct
980406	4.0068	4.1278	1269.98	1289.17	25/30	27/30
981406	4.0597	4.1582	1283.60	1288.97	26/30	27/30
982406	3.9731	4.1306	1337.45	1331.40	26/30	28/30

Setup: same code, image, target model, server settings, and protocol as the DFlash2 run above. The differences are:

• Speculative settings: --speculative-algorithm DSPARK with draft RadixArk/Qwen3.8-27B-DSpark, revision b9a5dbdf03 (DSparkDraftModel with a vanilla Markov head and a confidence head). Gamma 7 is read from the checkpoint, so the verify window is 8. SGLANG_RAGGED_VERIFY_MODE is left at its default, static.
• The two arms: they run identical commands, except that the block arm adds --speculative-use-block-verification. Every server logged DSpark draft proposal (greedy + sampling) folded into the draft cuda graph, so the sampling batches drafted from softmax(corrected_logits / T) and verified on the eager accept path, where the flag is wired. The folded in-graph accept serves all-greedy batches only.
• Audit: an independent pass over the raw per-request records reproduced every aggregate.

Sampler

No kernel change. The sampler cost is the block-mode cost measured in #42297. Paths that do not opt in are unchanged: the flag defaults to off, and greedy DFlash drafts never reach the block sampler.

Checklist

• Format code with pre-commit.
• Add registered unit tests.
• Update documentation.
• End-to-end acceptance-length and throughput comparison for DFlash2 and DSpark.
• Follow SGLang code style.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #37692039422
Latest PR Test (Extra): ⚠️ Not enabled -- add run-ci-extra label to opt in.
Latest PR Test (AMD ROCm 10): ❌ Run #37692039310

[maocheng23]

/tag-and-rerun-ci