Expand description
Shared helpers used by both Mamba3::forward
and Mamba3::step. They isolate three blocks
that previously appeared in both methods at different ranks:
- Trapezoidal discretisation:
dt,α,β,γ,da. - QK-norm + GQA expansion + per-(head, mimo-rank) bias on B / C.
- MIMO
Vconstruction: broadcast-multiplyxbymimo_x_hmp.
Each helper is generic over the rank D of the data tensors so a single
definition serves both the sequence-aware (forward) and single-token
(step) code paths.
Structs§
- Trapezoid
Coeffs - Output of
trapezoidal_coefficients.
Functions§
- build_
v_ with_ mimo - Build the MIMO value tensor
v = x ⊙ mimo_xwith broadcasting. - qk_
norm_ expand_ bias - QK-Norm → GQA-expand groups→heads → add per-(head, mimo-rank) bias.
- trapezoidal_
coefficients - Compute the trapezoidal discretisation coefficients from the raw
(data-dependent) projections. See the top-of-
mamba3.rsdocs for the formulas.