Expand description
Serial-over-chunks SSD with plain autodiff backward.
§Serial-over-chunks SSD (Mamba-3 double-SSD pathway)
The MIMO-first chunkwise scan as a serial loop over chunks, structured like
the five Mamba-2 kernels (super::super) but
generalised over the mimo_rank axis: the chunk and rank axes are fused into
a single length L·M for the intra-chunk products. The standard kernels
here are reused by both the γ-pass and the β-pass of the double-SSD
decomposition (the caller pre-scales v by γ or β and shifts the β inputs).
- K1
k1_ssd_chunk_cumsum— per-chunk cumulativeΔ·Adecays. - K2
k2_ssd_bmm— the intra-chunkC·Bᵀblock matmul (fusedL·M). - K3
k3_ssd_chunk_state— each chunk’s end-state contribution. - K4
k4_ssd_state_passing— the serial inter-chunk scan. - K5
k5_ssd_chunk_scan— combines intra- and inter-chunk parts intoy.
Produces identical values/gradients to super::minimal; SISO
(mimo_rank = 1) is the special case where the fused length equals the chunk
length. Gradients flow through plain autodiff.
Functions§
- k1_
ssd_ chunk_ cumsum - Compute the intra-chunk cumulative log-decay and per-chunk decay totals.
- k2_
ssd_ bmm - Compute the intra-chunk CB matrix on fused (mimo_rank-into-chunk_len) tensors.
- k3_
ssd_ chunk_ state - Compute the SSM state at the end of each chunk, assuming zero initial hidden state.
- k4_
ssd_ state_ passing - Propagate hidden state across chunk boundaries using a sequential scan.
- k5_
ssd_ chunk_ scan - Compute the chunk output by combining the intra-chunk (diagonal) and inter-chunk (off-diagonal) contributions.