[mickqian]

Summary

• remove the fixed 4 GiB runtime gate added by fix: skip unsafe automatic prefill graph capture #31204
• always capture an automatically selected prefill CUDA graph instead of silently falling back to eager execution based on free memory
• remove the defensive getattr and replace the obsolete threshold tests with a regression test that verifies capture still proceeds with 3.99 GiB free

Existing explicit disable and model/backend compatibility guards are unchanged.

Root cause

The memory gate used a model-agnostic fixed threshold after weights, KV cache, and eager buffers were allocated. That allowed default configurations to disable prefill CUDA graphs at runtime and introduced a silent performance regression for otherwise supported models.

Memory sizing should be handled by the configured cache/capture parameters. An automatically selected prefill graph should not be overridden by a hard-coded free-memory threshold.

Validation

• pre-commit run --files python/sglang/srt/model_executor/model_runner_components/cuda_graph_setup.py test/registered/unit/model_executor/test_prefill_cuda_graph_runner.py
• python3 -m pytest -q test/registered/unit/model_executor/test_prefill_cuda_graph_runner.py: 5 passed, 2 subtests passed
• H100 smoke with default sglang serve --model-path Qwen/Qwen3-8B (only host/port added):
  • resolved prefill.backend='breakable'
  • captured all 58 prefill buckets
  • prefill capture completed in 26.30s using 1.74 GiB
  • server reached health-ready and warmup logged cuda graph: True

Follow-up to #31204 and its review feedback.

---

CI States

Latest PR Test (Base): ✅ Run #31148378656
Latest PR Test (Extra): ❌ Run #31148378484

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[mickqian]

/tag-and-rerun-ci