Expand description
Rotary (RoPE) application: the mechanical pairwise rotation the abelian pathway factors into B/C. Rotary (RoPE) application helpers for the abelian rotation pathway.
Purely mechanical: given per-pair cumulative angles, rotate the paired channels of B/C. See the module header for why these angles are a state transition, not a positional encoding.
Functions§
- apply_
rope - Apply a rotary embedding (RoPE) to
xalong its last dimension. - apply_
rope_ partial - Apply RoPE to only the rotation-active entries of the last dimension; the
remainder passes through unchanged. Falls back to
apply_ropewhenrope_dim == state_rank(full RoPE).rope_dimmust be positive — a block that rotates nothing isRotationKind::Real1D, which never reaches here. - wrap_
angle - Reduce angles modulo
2πinto[−π, π], leaving the autodiff graph intact.