Expand description
§Mamba-2 chunkwise SSD algorithms
Three exact reformulations of the same Structured State Space Duality scan,
all agreeing on values and gradients (asserted by the ssd_path tests):
minimal— mostly batched matmuls + asegsummask; plain autodiff backward.serial— a serial loop over chunks (mirrors the Triton kernels K1–K5); plain autodiff backward.serial_recalculated— the same serial loop with a custom, memory-efficient backward that recomputes intermediates (saves ~⅓ training memory).
ssd_path holds the Mamba2SsdPath selector and the Mamba2SsdInput
bundle whose run() dispatches to one of the three.
Re-exports§
pub use serial_recalculated::Mamba2AutodiffBackendExt;pub use serial_recalculated::Mamba2BackendExt;pub use ssd_path::Mamba2SsdInput;pub use ssd_path::Mamba2SsdPath;
Modules§
- minimal
- Matmul/
segsumSSD with plain autodiff backward. - moments
- Closed-form per-token state moments (for state-PR diagnostics/penalties).
- serial
- Serial-over-chunks SSD with plain autodiff backward.
- serial_
recalculated - Serial-over-chunks SSD with a custom recompute backward. Serial SSD with a custom, memory-efficient backward.
- ssd_
path - The
Mamba2SsdPathselector andMamba2SsdInputdispatch bundle.