[BBuf]

Motivation

Support meituan-longcat/LongCat-2.0-FP8 in SGLang.

This PR includes the original LongCat 2.0 support from #30202 by @sjqgogogogo, plus the serving fixes found while validating the official FP8 checkpoint on an 8x B300 node.

Co-authored-by: sunjiaqi11 sunjiaqi11@meituan.com

Modifications

• Add LongCat 2.0 model/config support from support LongCat 2.0 #30202.
• Resolve LongCat HF configs through architectures when model_type is null.
• Map LongCat 2.0 oe_* ngram embedding config fields so model.ngram_embeddings.* weights are loaded.
• Align the ngram token-table path with the working implementation from support LongCat2.0 #30042 instead of the decode-only fast path that produced bad outputs.
• Rename the ngram token-table update skip field to make the chunked-prefill behavior explicit.
• Use consistent vocab-parallel TP settings for both LongCat word and ngram embedders.
• Populate ngram embedding metadata for warmup and CUDA graph dummy batches.
• Auto-resolve the FP8 GEMM runner backend for LongCat FP8 on Blackwell scale layouts.

Accuracy Tests

Hardware: 8x NVIDIA B300

Model: meituan-longcat/LongCat-2.0-FP8

Server command used for validation:

python3 -m sglang.launch_server \
  --model-path /mnt/local_disk/data/bbuf/models/LongCat-2.0-FP8 \
  --trust-remote-code \
  --host 0.0.0.0 \
  --port 13423 \
  --tp-size 8 \
  --ep 8 \
  --max-running-requests 64 \
  --mem-fraction-static 0.92 \
  --chunked-prefill-size 2048 \
  --nsa-prefill-backend fa3 \
  --kv-cache-dtype bfloat16 \
  --model-loader-extra-config '{"enable_multithread_load":true,"num_threads":12}'

Results:

• Generation check: A shop has 17 apples and sells 8. Then it buys 6 more. -> 15
• GSM8K 200 examples: 0.980
• GSM8K full set, 1314 examples: 0.958904109589041

CUDA graph was enabled and decode CUDA graph capture completed successfully.

Speed Tests and Profiling

The full GSM8K validation reported output throughput of about 1084 token/s. This was used as a serving validation, not as a formal throughput benchmark.

Checklist

• Format/diff check: git diff --check
• Python syntax check on changed files
• PYTHONPATH=python python3 test/registered/unit/model_executor/test_ngram_token_table.py
• GSM8K accuracy validation
• Full CI

---

CI States

Latest PR Test (Base): ⏳ Run #28853132542
Latest PR Test (Extra): ⏳ Run #28853132297

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!