Expand description
Softplus activation (fp16-aware).
Softplus activation: softplus(x) = log(1 + eˣ), a smooth ReLU.
Used to produce the strictly-positive discretisation step Δ (and, in
Mamba-3, the data-dependent A). The fp16 path uses the numerically-stable
identity softplus(x) = max(x, 0) + log(1 + e^−|x|) to avoid overflow in
eˣ; the wider formats use log1p(eˣ) directly.
Functions§
- softplus
- Applies the softplus function element-wise:
log(1 + eˣ).