MambaStateConfig#

class tensorrt_llm.llmapi.MambaStateConfig(
*,
periodic_snapshot_interval: ~typing.Annotated[int,
~annotated_types.Ge(ge=0)] = 0,
additional_snapshot_offsets_from_start: ~typing.List[~typing.Annotated[int,
~pydantic.types.Strict(strict=True),
FieldInfo(annotation=NoneType,
required=True,
metadata=[Gt(gt=0)])]] = <factory>,
additional_snapshot_offsets_from_end: ~typing.List[~typing.Annotated[int,
~pydantic.types.Strict(strict=True),
FieldInfo(annotation=NoneType,
required=True,
metadata=[Ge(ge=0)])]] = <factory>,
enable_branch_snapshot: bool = False,
)[source]#

Bases: StrictBaseModel

Configuration for reusable Mamba recurrent-state snapshots.

field additional_snapshot_offsets_from_end: List[Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0)])]] [Optional]#

Additional Mamba state snapshot offsets measured backward from the end of each prompt. An offset of 0 selects the prompt end. Offsets that do not resolve inside the prompt are ignored. These snapshots require KV cache manager V2.

field additional_snapshot_offsets_from_start: List[Annotated[int, Strict(strict=True), FieldInfo(annotation=NoneType, required=True, metadata=[Gt(gt=0)])]] [Optional]#

Additional Mamba state snapshot offsets measured from the start of each prompt. Offsets beyond the prompt length are ignored. These snapshots require KV cache manager V2.

field enable_branch_snapshot: bool = False#

Deprecated alias for kv_cache_config.block_reuse_config.enable_branch_snapshot. It takes effect only when block reuse is enabled.

field periodic_snapshot_interval: Annotated[int, Ge(ge=0)] = 0#

The number of tokens between periodic snapshots in the Mamba prefix cache. Periodic snapshots are disabled by default; set this to a positive value to enable them.

Constraints:

  • ge = 0

__init__(**data: Any) → None#

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.