Repository navigation
support LongCat 2.0 - #30202
support LongCat 2.0#30202sjqgogogogo wants to merge 1 commit into
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
| out_req_lens: torch.Tensor | ||
| # Mask marking chunked (not-yet-finished) prefill requests whose sampled | ||
| # pseudo next-token must NOT be written into the token table. | ||
| skip_mask: Optional[torch.Tensor] = None |
There was a problem hiding this comment.
nit: can we make this naming clearer? for example, ne_skip_token_table_update is clear for the use case
There was a problem hiding this comment.
Addressed in #30275 by renaming the NgramEmbeddingInfo field to skip_token_table_update and keeping ScheduleBatch.ne_skip_token_table_update as the scheduler-facing field.
| num_embeddings=self.exclusive_oe_embedder_size_sums[-1], | ||
| embedding_dim=oe_hidden_dim, | ||
| enable_tp=is_dp_attention_enabled(), | ||
| enable_tp=True, |
There was a problem hiding this comment.
why only change this for oe_embeder while word_embeder not changed?
There was a problem hiding this comment.
Addressed in #30275 by making word_embeder and oe_embeder use the same vocab-parallel TP setup (use_attn_tp_group=is_dp_attention_enabled()), matching the normal LongCat embedding path instead of only changing the OE embedder.
|
Thanks @sjqgogogogo. Closing this because it has had no updates in 91 days. Reopen it if the work is still relevant. Some directories moved recently, so an older branch may need retargeting: |
Motivation
Modifications
Accuracy Tests
Speed Tests and Profiling
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ❌ Run #28766023588
Latest PR Test (Extra): ❌ Run #28766023574