[vlserov]

Motivation

The LoRA linear classes repeatedly implemented the same logic as the base linear classes. To avoid conflicts and simplify future maintenance, the linear class logic needs to be refactored.

Visualisation comparing LoRA linear classes and base linear classes:

Modifications

• A base linear class ParallelLinear was created, which implements common behaviour for all linear classes.

• Other linear classes inherit from ParallelLinear and only re‑implement parts that differ from the base implementation.

• The LoRA base linear class BaseLayerWithLoRA implements a generic forward method. Each inheriting class specifies the behaviour of its respective base linear layer while leveraging the base LoRA implementation.

Accuracy Tests

Server launch and request

sglang serve --model-path Tongyi-MAI/Z-Image-Turbo --model-type diffusion --dit-cpu-offload false --num-gpus 1 --port 8080 --host localhost

curl -sS -X POST "http://localhost:8080/v1/images/generations" -H "Content-Type: application/json" -H "Authorization: Bearer sk-proj-1234567890" `
    -d '{ `
        "prompt": "At night, while their owner sleeps in a bedroom, three cats march in loudly playing tiny brass instruments, then abruptly file out.", `
        "size": "1024x1024", `
        }'

curl -sS -X POST "http://80.48.5.153:30040/v1/set_lora" -H "Content-Type: application/json"  `
    -d '{ `
        "lora_nickname": "lora_name", `
        "lora_path": "Ttio2/Z-Image-Turbo-pencil-sketch", `
        "target": "all", `
        }'

Output results:

Speed Tests and Profiling

platform linux -- Python 3.11.14, pytest-9.0.2, pluggy-1.6.0
configfile: pyproject.toml
plugins: cov-7.1.0, anyio-4.12.0
collected 40 items

python/sglang/multimodal_gen/test/unit/test_lora_peft.py ......... [ 22%]
python/sglang/multimodal_gen/test/unit/test_lora_pipeline.py ........ [ 42%]
python/sglang/multimodal_gen/test/unit/test_fused_lora_compose.py ........... [ 70%]
python/sglang/multimodal_gen/test/unit/test_lora_merge_cache.py ..... [ 82%]
python/sglang/multimodal_gen/test/unit/test_lora_format_adapter.py . [ 85%]
python/sglang/multimodal_gen/test/unit/test_lora_inference_mode.py .. [ 90%]
python/sglang/multimodal_gen/test/unit/test_lora_commit_as_base.py .... [100%]
=========================== 40 passed, 17 warnings in 22.64s ===========================

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ❌ Run #40185619515
Latest PR Test (Extra): ❌ Run #40185619518
Latest PR Test (AMD ROCm 10): ❌ Run #40185619506

[ping1jing2]

please update the description, then click ready to review

[mickqian]

/tag-and-rerun-ci

[vlserov]

/rerun-failed-ci

[mickqian]

CI follow-up on exact head 0362801: 2-GPU partition 0 fails wan2_2_t2v_a14b_lora_2gpu consistency and LoRA basic API validation (SSIM 0.7091 vs >=0.95; PSNR 17.0777 vs >=22.2). 1-GPU partition 3 fails LTX-2.3 HQ consistency as well as E2E latency (25.884s vs limit19.968s). Both suites report 1 failed/8 passed. These are not merely runner-timeout failures, so I have not rerun them or changed GT/thresholds.

One concrete regression candidate in the refactor: the shared LoRA forward computes input_parallel = base_layer.parallel_input(input), but then feeds the original input to _compute_lora_delta. For RowParallelLinear with input_is_parallel=False and TP>1, slice_lora_a_weights still returns a TP-sharded A, whereas the previous row-specific forward used input_parallel for this multiplication. Please use the same sharded input for the LoRA path and add a real row-parallel unmerged-LoRA regression covering input_is_parallel=False. This source-level mismatch is not yet proven to explain the observed CI image difference; also validate the merged and output-offset paths before updating any golden output. I have no push access to the head repository and have not modified the branch.

[vlserov]

@mickqian thank you for the review. Comment has been resolved.

[mickqian]

CI follow-up on exact head 8ab1a64:

• H100 single-GPU partition 3: fast_hunyuan_video and fastwan2_2_ti2v_5b recover on failed-item retry. The final remaining failure is ltx_2_3_hq_pipeline consistency (minimum SSIM 0.9507, PSNR 32.6642; maximum MAD 2.6987), against the existing per-case gates and pinned GT 252710158cd4c74b7604808a385e93f7bced6d28. This is not merely the earlier E2E failure; consistency must not be retried away or regenerated just to pass.
• B200, runner b200-cirrascale1-4567: final retry still fails Flux1 NVFP4 E2E 1432.4429ms > 1045.8875ms and Qwen2512 NVFP4 E2E 18734.9121ms > 12062.5750ms. Earlier failures in other cases recover, so the accumulated failure lines are not the final failing set.
• All three dual-GPU partitions are health-check cascades from these roots. The coverage check consequently reports all 28 dual-GPU cases missing; this does not establish a partition-registration bug or validate the new head's dual-GPU LoRA path.

The PR changes the shared linear/LoRA execution path; the logs alone do not distinguish a numerical/performance regression from environment effects. Please compare the parent and this exact head on the same runner/environment with unchanged configuration and gates. I have no push access to the head repository, have not modified the branch, and have not rerun the non-retryable consistency failure.

[mickqian]

CI follow-up: GitHub currently reports a merge conflict with main on exact head 8ab1a64. Please merge the current target branch with a normal merge commit and resolve the conflicts, then validate the new head. I do not have push permission to this fork; I have not edited it or requested a CI rerun. This is a new mergeability blocker, separate from existing CI results.

[github-actions]

Required checks failed on f7a9523; this PR cannot be merged until they pass.

PR title (run)

PR title must use "[Type] scope: title" or "[Type] [Hardware] scope: title". Type must be one of: Feat, Fix, Refactor, Perf, Doc, Test, CI, Chore. Scope is required and may be any module or model name, with unrestricted letter case (letters, numbers, dots, underscores, slashes, or hyphens). Hardware is optional; use a single tag without spaces. Use uppercase hardware tags (AMD, NVIDIA, NPU, MI300X), except for exactly these four spellings: Intel, Apple, Huawei, Ascend. Reverts must use GitHub's default format: Revert "original title". Use one space between fields and after the colon, with a non-empty title and no leading or trailing whitespace. Examples: "[Fix] attention: Fix KV cache indexing", "[Feat] qwen3: Add model support", "[Perf] [AMD] flash-attn: Optimize decode performance".
Edit the title; the check reruns on its own.