[huangxiaojun15]

Motivation

Move the SGLang NPU stack from CANN 9.0.0 / Python 3.11 to CANN 9.1.0 / Python 3.12, and align the
device matrix with what 9.1.0 actually ships.

• CANN 9.1.0 is the current Ascend toolkit release, and the prebuilt base images
  (quay.io/ascend/cann:9.1.0-<device_type>-ubuntu22.04-py3.12) are published for both a3 and 950.
• The torch_npu and sgl-kernel-npu builds we consume now target CANN 9.1.0 on Python 3.12:
  torch_npu 2.10.0.post6 is published on the Ascend devcloud PyPI index, and
  sgl-kernel-npu 2026.9.0.post5 ships py312-cann9.1.0 artifacts for a3 / 950.
  The cann9.0.0 artifacts of the same release only exist for Python 3.11, so the image, the CI
  install script and the toolkit version have to move together.
• A2 (910b) is dropped from the image matrix. The nightly/release pipelines now publish only
  a3 (A3) and 950 (A5) images, and the A2-only CI jobs that consumed a 910b image are removed.
  A2 hardware PR tests that run directly on the ascend-ci/cann container are not affected.
• Keeps the published image tags (lmsysorg/sglang:<version>-cann9.1.0-<device_type> for releases,
  lmsysorg/sglang:main-cann9.1.0-<device_type> for nightly) and the CI that consumes them on the
  same baseline, instead of testing a different toolkit than users pull.

Modifications

Docker image — docker/npu.Dockerfile

• Base image moves to quay.io/ascend/cann:${CANN_VERSION}-${DEVICE_TYPE}-${OS}-py3.12 with
  CANN_VERSION=9.1.0 and PYTHON_VERSION=py3.12 by default.
• MemFabric / MemCache are selected per device:
  950 fetches the 1.2.1 wheels from the sglang-npu OBS bucket (sha256 verified) and runs
  mfcli kernel install; a3 installs memfabric-hybrid==1.1.5 / memcache-hybrid==1.1.5 from PyPI.
• memfabric-zbal is pinned per device: 1.2.21004.post1 for 950, 1.1.3 for a3.
• PyTorch pairing updated: torch 2.10.0 + torch_npu 2.10.0.post6 (installed from the Ascend
  devcloud index, replacing the pinned GitCode wheel), triton-ascend 3.2.2 (aarch64 wheel from the
  Ascend OBS, x86_64 wheel from the triton-ascend GitHub release) instead of 3.2.1.dev20260530.
• sgl-kernel-npu bumps to 2026.9.0.post5; custom-ops / ops-transformer / sgl-kernel-npu are
  downloaded as cann${CANN_VERSION} / py312 artifacts and installed into
  ${ASCEND_HOME_PATH}/opp, with the CANN / ATB set_env scripts sourced first.
• ASCEND_HOME_PATH and LD_LIBRARY_PATH are now set explicitly in the image.
• Adds ModelScope / EvalScope installation (versions overridable via build args, empty = latest).

CI dependency install — scripts/ci/npu/npu_ci_install_dependency.sh

• Same dependency set and versions as the image, but every pin is overridable from the environment
  (CANN_VERSION, PYTORCH_VERSION, TORCH_NPU_VERSION, SGLANG_KERNEL_NPU_TAG, MF_*, ZBAL_*).
• Same per-device MemFabric / MemCache / zbal logic as the Dockerfile, including the sha256 check of
  the OBS wheels; a3 uses the PyPI 1.1.5 wheels.
• ASCEND_HOME_PATH / LD_LIBRARY_PATH are derived from CANN_VERSION, and a source_env helper
  skips set_env files that do not exist instead of aborting.
• Installs torch 2.10.0 + torch_npu 2.10.0.post6 and triton-ascend 3.2.2 (per architecture),
  then unpacks the py312-cann9.1.0 sgl-kernel-npu / custom-ops / ops-transformer archives.

Workflows

• .github/workflows/pr-test-npu.yml and .github/workflows/diffusion-ci-gt-gen-npu.yml: NPU test
  containers move to ascend-ci/cann:9.1.0-a3-ubuntu22.04-py3.12.
• .github/workflows/pr-test-npu.yml: removes the A2 (910b) job and the CANN_image_910b output,
  and drops the corresponding entry from the downstream needs: chains.
• .github/workflows/release-docker-npu.yml: matrix becomes cann_version: ["9.1.0"],
  device_type: ["a3", "950"], so tag releases publish v<version>-cann9.1.0-{a3,950}.
• .github/workflows/release-docker-npu-nightly.yml: rebuilds the image as four single-architecture
  builds on native runners (a3/950 x amd64/arm64) plus a merge job that assembles the
  multi-arch manifest with docker buildx imagetools, replacing the single QEMU multi-arch job. The
  published tag is main-cann9.1.0-<device_type> and GHA layer cache is enabled per device/platform.
• .github/workflows/full-test-npu.yml and .github/workflows/nightly-test-npu.yml: default test
  images move to main-cann9.1.0-a3 / main-cann9.1.0-950, and the image_a2 (910b) input plus the
  full-4-npu-a2 job are removed.
• .github/workflows/_npu-single-node-test-stage.yml: default image becomes main-cann9.1.0-a3.
• .github/workflows/_npu-pr-test-stage.yml and .github/workflows/coverage-collection-npu.yml:
  description/comment updates that follow the A2 removal.

Accuracy Tests

No model or kernel source is changed; the PR moves the toolkit version, the images and the CI that
consumes them. The NPU operators are rebuilt against the new toolkit, so the NPU accuracy suites are
re-run on the CANN 9.1.0 image before merge.

Model	Dataset / metric	CANN 9.1.0	CANN 9.0.0 baseline

Speed Tests and Profiling

The toolkit bump changes the compiled operators, so throughput/latency is compared on both toolkits
with the same hardware and workload.

Model	Hardware	Input / Output len	Concurrency	CANN 9.1.0 (output tok/s)	CANN 9.0.0 (output tok/s)

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): 🚫 Run #35966604642
Latest PR Test (Extra): ❌ Run #35966604276
Latest PR Test (AMD ROCm 10): ⏳ Run #35966604725

[sglang-npu-bot]

/tag-run-ci-label

[huangxiaojun15]

/rerun-failed-ci

[huangxiaojun15]

/rerun-failed-ci

[huangxiaojun15]

/rerun-failed-ci