Skip to content

[RFC #580][Qwen3-8B Dense][ROCm MI300X][Stage 0] Inventory strict callsites and select the MVP route #581

Open

Description

@inaniloquentee

Parent RFC: #580: Consistency-preserving profile-guided operator dispatch.

This issue implements Stage 0 of the RFC. The first MVP is limited to one existing Qwen3-8B Dense ROCm workload and one operator/configuration family. It does not expand the supported hardware/topology matrix.

Scope

Documentation and read-only inventory only. Trace the actual supported ROCm production path through VIME, vLLM rollout, and Megatron training on MI300X/gfx942. Qwen3-8B Dense on the documented eight-GPU MI300X/gfx942 ROCm reference path, with its pinned ROCm profile, VIME/vLLM/Megatron companion sources, and training TP4/CP2 plus rollout TP4/CP1 where the profile resolves those settings. CUDA/H100 evidence is not ROCm evidence.

Deliverables

  • Map each relevant semantic operator/region to its actual production callsites, requested implementation, observed implementation, forward/backward owner, graph/capture boundary, and provenance source.
  • Inventory existing implementations/configurations already loadable in the frozen environment; do not count unrelated test-only or unsupported candidates.
  • Select exactly one MVP operator/region only if at least two realistic candidates can be compared under the same strict contract. Prefer the region with the smallest observable boundary and strongest evidence; record why it was selected.
  • Define the exact Qwen3 workload/profile, shapes and guards, weight/token identity, numerical boundaries, certificate fields, memory budget assumptions, and static baseline.
  • If no pair of potentially bitwise-compatible candidates exists, record a no-go with evidence and recommend one narrowly scoped candidate-enablement follow-up. Do not claim the profiler can create alternatives.

ROCm-specific scope

  • Treat the candidate family as launch/tile configurations of the existing rlkernel.det_gemm.triton_mfma_rocm.v1 arithmetic contract, not MFMA versus the numerically different tree backend.
  • Map its M,N,K, operand-layout, training/backward and rollout callsites, and the current select_config() static baseline.
  • Use reports/archive/benchmarks/pr396_rocm_mi300x/report.md and the gfx942 bitwise schedule tests as prior evidence, not as a replacement for the new exact workload/profile measurements.

Acceptance criteria

  • Inventory points to the real rollout and training execution path; documentation-only model/reference coverage is explicitly labeled.
  • Exactly one MVP region and bounded initial workload are named, or an evidence-backed no-go is recorded.
  • All unsupported shapes, phases, layouts, dtypes, and topologies are listed; no new support is implied.
  • No runtime behavior or numerical contract changes in this stage.

Non-negotiable consistency requirements

  • Training independently recomputes selected LogP; aligned active tokens have zero raw-byte/bit mismatches against the corresponding rollout under the same weights and scoring contract.
  • Candidate substitution preserves the applicable strict numerical contract. Deterministic status, allclose, a profile score, or a performance win is not proof of interchangeability.
  • Existing FP32-reference accuracy thresholds, forward/backward obligations, invariance checks, and applicable model-chain gates remain unchanged.
  • No rollout-score reuse, silent fallback, tolerance relaxation, approximate route, or catch-OOM-and-retry after state mutation.
  • Missing certification or an infeasible memory budget fails closed. The static certified baseline remains the default.

Out of scope

No live-request autotuning, cross-vendor claims, unvalidated topology, additional operator family, or default-on behavior in this phase.

References

Activity

  1. added
    component: kernelsTasks involving the development of CUDA and Triton underlying operators
    component: alignmentTasks involving RL loss functions such as DPO and GRPO, and mathematical alignment logic
    type: designIssues requiring in-depth discussion of architecture design
    on Oct 24, 2026
  2. changed the title [-][RFC #580][Qwen3-8B Dense][Stage 0] Inventory strict callsites and select the MVP route[/-] [+][RFC #580][Qwen3-8B Dense][ROCm MI300X][Stage 0] Inventory strict callsites and select the MVP route[/+] on Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    component: alignmentTasks involving RL loss functions such as DPO and GRPO, and mathematical alignment logiccomponent: kernelsTasks involving the development of CUDA and Triton underlying operatorsqwentype: designIssues requiring in-depth discussion of architecture design

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions