Skip to main content

Module cache

Module cache 

Source
Expand description

The single-SSD cache (same fields as double-SSD, different ssm semantics).

§Mamba-3 Single-pass SSD Inference Cache

The cache used by crate::mamba3::mamba3::Mamba3::forward_single_ssd (the single-pass SSD algorithm — see the Triton SISO and Tilelang MIMO reference kernels). The four tensor fields mirror those of [Mamba3Cache] but their SSM accumulator carries different semantics:

  • [Mamba3Cache]: ssm_bhpr holds the double-ssd trapezoidal hidden state hₜ = αₜ hₜ₋₁ + βₜ Bₜ₋₁ ⊗ xₜ₋₁ + γₜ Bₜ ⊗ xₜ.
  • [Mamba3SingleSsdCache]: ssm_bhpr holds the trapezoid accumulator h'ₜ defined by h'ₜ = αₜ h'ₜ₋₁ + scaleₜ Bₜ ⊗ xₜ, where scaleₜ = γₜ + (1 − λₜ₊₁) · Δₜ₊₁. The single-ssd form gives the correct output yₜ = Cₜᵀ h'ₜ for all positions except the diagonal (s = t), which is patched by an explicit γₜ · (Cₜᵀ Bₜ) · xₜ correction term in the kernel.

Because the two accumulators differ, the two caches are not interchangeable. The distinct type prevents accidentally feeding a forward_double_ssd cache into forward_single_ssd (or vice versa) mid-sequence — that would silently corrupt state.

Structs§

Mamba3SingleSsdCache
Mutable state for a single Mamba-3 layer running the single-ssd form algorithm.
Mamba3SingleSsdCacheConfig
Configuration / factory for a single Mamba3SingleSsdCache.
Mamba3SingleSsdCacheRecord
The record type for the module.
Mamba3SingleSsdCacheRecordItem
The record item type for the module.
Mamba3SingleSsdCaches
A collection of per-layer single-ssd form caches for a complete Mamba-3 network.
Mamba3SingleSsdCachesConfig
Configuration / factory for Mamba3SingleSsdCaches.
Mamba3SingleSsdCachesRecord
The record type for the module.
Mamba3SingleSsdCachesRecordItem
The record item type for the module.