Givens & Delta: a composer deep dive¶

Orientation¶

GivensMinGRU and DeltaMinGRU are this library's two composers for extract-then-compose stacks, the upper layer that takes an operation already extracted by a lower layer (typically SignedMinGRU) and accumulates it non-commutatively. Both solve the same placement problem, on tasks like S3-hier, with different mechanisms and different resource shapes. GivensMinGRU composes rotations across a block_size-dimensional block: an exactly special-orthogonal transition held at a small, fixed per-token state (the standard \(64\) elements at this repository's d_model). DeltaMinGRU composes a delta-rule associative memory (a per-head \(d_k \times d_v\) matrix updated by generalized-Householder rank-one corrections), whose state is an independent capacity knob (n_heads, nh, d_k, d_v) that can grow far past \(64\) elements per token, and whose forward runs the DeltaNet literature's chunked-WY parallel form rather than an associative scan.

This article explains both mechanisms (why each per-token map exists and how it is parameterized) and the evidence that separates them: fit reliability at matched state, cost and memory as state grows, and the two-axis recommendation that evidence now supports. For the constructor signatures and keyword arguments, see Reference: mixers. For the choice between the two composers on a real task, see How-to: choose a mixer.

The headline, upfront. State free to grow → DeltaMinGRU at its native size is the promoted default: the most reliable trained composer measured on both hardware strata (CPU, pinned torch 2.5.1: 12/12 fits at state 1024; GPU, torch 2.8.0+cu128: 35/36 fits under torch.compile), with per-step cost and memory that stay nearly flat as state grows. State that must stay small (a matched 64-element budget) → GivensMinGRU is decisively more reliable on the larger-sample GPU stratum (\(25/36\) against delta's \(10/36\), Fisher \(p = 0.00084\)), and its small-state fit cohort holds up better at extreme extrapolation length (\(16\times\) train length, fit-only: givens \(0.679\) / delta@64 \(0.699\), both ahead of delta@1024's \(0.570\)). Neither composer is "more accurate" in an unqualified sense. See Composer reliability below for why that framing is wrong. The rest of this article is the mechanism and the evidence behind the headline.

The rung it enriches¶

The base minGRU recurrence is a first-order linear scan \(h_t = a_t \odot h_{t-1} + b_t\) whose transition \(a_t\) is diagonal and, in the log-space parameterization, confined to \((0, 1)\). Two structural limits follow, and both are expressivity limits rather than optimization difficulties: memory can only decay monotonically, and the transition is diagonal, hence commutative, so the running product is invariant to input reordering. SignedMinGRU lifts the first limit by admitting \(a_t \in (-1, 1)\). Neither the base nor the signed variant lifts the second: diagonal transitions commute under scalar multiplication, so permutation composition (where order changes the answer) stays out of reach at any width.

Lifting commutativity requires a non-diagonal transition. RotationMinGRU makes each per-block transition a full \(2 \times 2\) affine map \(M_t = R(\theta_t)\,\mathrm{diag}(1, \tanh u_t)\), an \(O(2)\)-valued family that generates the dihedral group. Since \(D_3 \cong S_3\) (the smallest non-abelian group) embeds in \(O(2)\), one layer can exactly represent the \(S_3\) running-product automaton. GivensMinGRU occupies this same rung with a more expressive per-token map: instead of one plane it composes rotations across many planes of a larger block, reaching all of \(SO(k)\) in the limit while keeping the per-token cost linear in width. It is emphatically not a fifth rung above rotation. The formal boundary (that all four mixers are input-dependent, state-independent linear transitions and therefore live in \(\mathrm{TC}^0\), unable to do the unbounded state tracking a single nonlinear GRU layer can) is unchanged by the block size. See experiments/TECHNICAL_REPORT.md §1–2 for the full expressivity argument.

DeltaMinGRU shares that same formal ceiling (its transition is input-dependent and state-independent, so it is \(\mathrm{TC}^0\) too), but it is not a Givens machine at a larger block: instead of composing orthogonal rotations, it composes rank-one Householder-style corrections into an unconstrained-norm memory matrix, the DeltaNet/DeltaProduct transition rule. It buys non-commutative composition the same way the rotation family does, through a non-diagonal transition, but the map itself is a different shape with a different, and independently adjustable, state budget. The next section covers that mechanism before the article turns to the evidence that separates the two composers.

The brick-wall Givens parameterization¶

What a Givens rotation is¶

A Givens rotation is the smallest possible rotation. It acts on one coordinate plane of a \(k\)-dimensional space and leaves the other \(k-2\) directions fixed: the identity matrix everywhere except for \(\cos\)/\(\sin\) entries in the two rows of the chosen plane. It is the atom of orthogonal structure, exactly orthogonal at every parameter value with no normalization step, determinant \(+1\), one angle per plane. In this sense RotationMinGRU is already a Givens machine at \(k = 2\): one plane, one input-dependent angle per token.

Composing planes in a brick-wall mesh¶

GivensMinGRU composes many Givens rotations per token. At the default block_size \(= 8\), rounds \(= 3\), each per-token transition is a product of three brick-wall layers of Givens rotations across an \(8\)-dimensional block:

• round 0 pairs the planes \((0,1),(2,3),(4,5),(6,7)\);
• round 1 pairs the staggered planes \((1,2),(3,4),(5,6),(7,0)\);
• round 2 repeats round 0's pattern.

All angles are emitted by one linear head from the current input. The crucial structural fact is what commutes and what does not. Disjoint planes within a round share no coordinate, so they commute and can be applied in any order; the mesh is embarrassingly parallel inside a round. The stagger between rounds is what couples all eight dimensions: round 1's plane \((1,2)\) shares a coordinate with round 0's \((0,1)\) and \((2,3)\), so the composition of round 0 then round 1 does not factor into independent planes. This is the entire point of the mesh, and, as the rounds ablation below shows, it is measurably the source of the variant's advantage.

Products of enough Givens rotations reach all of \(SO(k)\), so three rounds are a deliberate compute budget, not a representational ceiling. The result stays continuous, stays exactly special-orthogonal (no normalization, no snap grid), stays parallel through the same associative scan generalized to \(k \times k\) blocks (matrix_affine_scan), and keeps the standard \(64\)-element per-token state at the repository's d_model.

Why brick-wall, specifically¶

The brick-wall mesh is not an ad-hoc choice. It is the standard construction from the orthogonal/unitary-RNN literature: the EUNN factorization of Jing et al. (ICML 2017), built on the rectangular interferometer mesh of Clements et al. (2016), with the Givens rotation as described in Golub & Van Loan, Matrix Computations. The mesh's appeal is that it reaches a wide swath of \(SO(k)\) using only \(O(k)\) angles per layer (one per plane) rather than the \(O(k^2)\) parameters a dense orthogonal transition would need, and it does so with a fixed, hardware-friendly connectivity pattern. What is specific to this variant is not the factorization but the two things layered on top of it: the angles are input-dependent per token (a fresh transition each step, emitted from \(x_t\)), and the contribution is the measured trainability result below, not a claim about the mesh itself.

Practical shape¶

GivensMinGRU carries three linear heads (\(\theta\), \(z\), \(h\)) against RotationMinGRU's four and SignedMinGRU's three, worth remembering in parameter-matched comparisons. It owns its initial state \(h_0\) as an intrinsic learned parameter with no learnable_h0 flag: a zero state has no orbit under the group action, so it cannot demonstrate tracking. It requires hidden_size divisible by an even block_size. Numerical agreement between the parallel scan and the sequential step path is \(\approx 10^{-5}\) at \(T=128\) in fp32, slightly looser than the other mixers because a product of rounds \(k \times k\) blocks per step accumulates more matrix arithmetic than a single \(2 \times 2\). Full signatures and validation rules are in Reference: mixers.

The delta-rule composer: chunked-WY as a parallel associative memory¶

Where GivensMinGRU composes orthogonal rotations, DeltaMinGRU composes a delta-rule associative memory: the DeltaNet (Yang et al. 2024, arXiv:2406.06484) / DeltaProduct (Siems et al., NeurIPS 2025, arXiv:2502.10297) transition rule, packaged in this library as mixer="delta". Its per-token state is not a block_size-dimensional vector but a per-head \(d_k \times d_v\) matrix, flattened to (B, n_heads * d_k * d_v): capacity is set by n_heads, nh (corrections per token), d_k, d_v (independent of d_model), so the state budget is a first-class knob rather than a fixed \(64\) elements tied to the stack width. Each token applies nh generalized-Householder rank-one corrections to the memory matrix, the mechanism DeltaNet/DeltaProduct use to track state beyond what a diagonal or orthogonal transition can represent. Unlike the other mixers here, DeltaMinGRU rejects any decay= other than None (folding per-token time decay into the delta-rule composition is unimplemented), and its step returns (y_t, h_t) rather than a single tensor, since the readout is not the carried state.

Why chunked WY, not a sequential loop or a naive scan¶

A delta-rule update is sequential by construction (each rank-one correction depends on the memory state the previous token left behind), which looks like it forfeits the parallel-training property every other mixer in this library keeps. The DeltaNet literature's chunked WY representation removes that: within a chunk, it reformulates the sequence of corrections as a triangular solve plus a small number of batched matrix products, so the whole chunk resolves without a token-by-token loop, and chunks compose sequentially only at chunk boundaries (chunk_size is a performance-only knob; results are invariant to it up to numerical noise). This is what DeltaMinGRU.forward implements; the sequential per-token loop survives as step and as the correctness oracle the parallel path is checked against.

The chunked-WY form is not merely "also parallel": on CPU it is the cheapest forward path measured anywhere in this library, cheaper even than mixers whose per-token transition is far smaller. Under the pinned bench (experiments/bench/delta_paths.md; torch 2.5.1), one uncontended forward+backward step at \(B=128\), \(T=64\) costs \(0.0577\)s for chunked-WY against \(0.1617\)s for the sequential step-loop and \(2.0514\)s for a naive per-token reduction through the general \(k \times k\) scan machinery: chunked-WY beats the naive parallel reduction by \(35.55\times\) and the sequential oracle by \(2.80\times\); the gap widens at \(T=1024\) (\(1.4541\)s against \(19.9742\)s and \(71.7562\)s, \(49.35\times\) and \(13.74\times\)). Sequentiality was never what made a delta-rule step expensive on CPU: the naive parallel reduction is slower than even the sequential loop, and the chunked-WY transform is what actually closes the gap.

Flat cost and memory as state grows¶

Because delta's per-token state is decoupled from d_model, cost and memory can be measured directly against state size, a question GivensMinGRU, whose state is tied to d_model, cannot pose the same way. The mechanism × state scaling probe (experiments/bench/scaling_frontier.md; torch 2.5.1, uncontended, \(B=128\), \(T=64\), forward+backward, median of 5 timed steps) sweeps both mechanisms' per-token state from \(64\) to \(4096\) elements:

Delta's cost and memory are nearly flat across a \(64\times\) state range; Givens scales steeply, and its \(4096\)-element row nearly saturates a 32 GB machine. At matched state \(1024\), delta is roughly \(244\times\) faster per step and \(14\times\) lighter; at \(4096\), roughly \(916\times\) faster and \(30\times\) lighter. This is the structural reason "state free to grow" favors delta: growing delta's state is nearly free, growing Givens' state is not; the frontier, not a single-point measurement, is what the state-budget axis of the recommendation rests on.

What the richer map demonstrates against diagonal RNNs¶

The reason a non-commutative transition matters is best seen on the task that isolates it. Diagonal transitions commute, so a stack of them yields the same final state under any reordering of the inputs. A task whose labels change under reordering (composing permutations, where doing \(g\) then \(h\) differs from \(h\) then \(g\)) therefore cannot be represented by any diagonal variant at any width. This is not a training gap that more depth or more steps would close; it is a representability wall. The rotation family is what climbs over it.

The evidence base is the S3-hier task, an extract-then-compose problem harder than the plain S3 probe. The group operation is hidden inside a pair of sub-tokens drawn uniformly from \(\{0,\dots,5\}\): each consecutive pair \((x_{2k}, x_{2k+1})\) selects a generator \(g = \mathrm{LATIN}[a, b]\), composed onto the running \(S_3\) product when the pair completes (chance \(\approx 1/6 \approx 0.167\)). LATIN is a fixed \(6 \times 6\) Latin square, verified non-isotopic to both groups of order six (\(\mathbb{Z}_6\) and \(S_3\)), so a single sub-token carries no information about the generator and no rotation layer can absorb the lookup by relabeling its angle assignment. Extraction is genuine work that must precede composition, which is why the promoted configuration is a two-layer stack: a signed extractor below, a Givens composer above. The task construction and the isotopy verification are documented in probes.py and experiments/TECHNICAL_REPORT.md §3.1.

Two properties make this evidence unusually trustworthy. First, S3-hier is a reliability finding, not an exactness one: because even an unconstrained GRU sits at chance here in budget (\(0.171\) at \(T=1024\)), the task measures which inductive bias trains reliably under the \(1600\)-step budget, not what is representable. Composers differ in how often training finds a working solution, not in how well found solutions generalize. Second, the promoted code path is provably the evidence path.

The seed-0 provenance transfer¶

The multi-seed campaign that produced the GivensMinGRU numbers was trained through the lab harness (experiments/hetero_lab.py), not through the shipped MinGRUStack. To close that gap, one probes.py run of the promoted registry row minGRU-hetero-sg8 on S3-hier, seed 0, under the CKPT=1 best-val@128 protocol at MAX_STEPS=1600, was compared field-by-field against the recorded lab row, under the torch==2.5.1 evidence pin (round givens-promotion-replication-01 in experiments/EXPERIMENTS.md):

Exact match on every metric, no tolerance widening. This transfers the pooled twelve-seed evidence onto the promoted min_gru.py code path without rerunning the campaign, and it is the single row a skeptic should check first: it demonstrates that the number the README reports for the shipped class is the number that was actually measured. The 1.000 / 0.9941 / 0.9105 / 0.6619 shape across \(T = 64/256/512/1024\) is also the microcosm of the whole story: a near-perfect in-distribution fit that decays with length, which the trade-offs section explains.

The map-richness result¶

At matched \(64\)-element per-token state, \(8\)-dimensional Givens blocks fit S3-hier on 8 of 12 seeds against 1 of 12 for continuous \(2\)D rotation blocks (Fisher exact \(p \approx 0.0094\)). The separation is threshold-robust: at fit thresholds \(\{0.98, 0.99, 0.995\}\) the counts read \(9/12\), \(8/12\), \(8/12\) against \(1/12\) at all three (\(p = 0.0028, 0.0094, 0.0094\)). Pooled across all twelve seeds, the signed → givens8 profile is \(0.949 / 0.885 / 0.787 / 0.613\) at \(T = 64/256/512/1024\); among the seeds that fit, it holds \(0.927\) at \(T=512\) and \(0.733\) at \(T=1024\) (best seed \(0.956 / 0.812\)). The full cross-mechanism table, including the delta-rule comparison, is in experiments/TECHNICAL_REPORT.md §4.4 and the README's S3-hier section.

Reliability is separable from generalization quality. Seeds that fit generalize equally under either mechanism, fit-only means of \(0.733\) (Givens) against \(0.739\) (a small delta-rule composer) at \(T=1024\), so the mechanisms differ in fit rate, not in the quality of the solutions they find. That is the honest shape of the win: not "Givens generalizes better," but "Givens trains to a working composer far more often at matched state."

The rounds ablation: map richness versus block size¶

The map-richness comparison on its own moves two things at once: per-token map expressivity (\(SO(8)\) versus \(SO(2)\)) and within-scan block connectivity (eight coupled channels per block versus two). An external adversarial review of the technical report flagged this exact confound: "map richness raises fit rate" was supported only correlationally. The rounds ablation (round hetero-loop-19-rounds, n=12 per arm, same stack and protocol) resolves it by holding block_size \(= 8\) fixed and varying rounds:

At rounds \(= 1\) the brick-wall mesh is four disjoint, commuting \(2\)D planes inside the \(8\)D block: thirty-two planes per token, plane-count-matched to the \(2\)D composer's thirty-two blocks, but with the larger coupled state block. It fits 0/12: no better than the \(2\)D composer's \(1/12\) (Fisher \(p = 1.0\)) despite the bigger block. This refutes the block-size hypothesis outright: a larger block of commuting planes buys nothing. The single staggered coupling layer of rounds \(= 2\) (the one layer that breaks within-block commutativity) recovers most of the effect (\(6/12\); \(p \approx 0.014\) versus rounds \(= 1\)), and the third round adds a statistically inseparable increment (\(8/12\); \(p \approx 0.68\) versus rounds \(= 2\)). The endpoint separation \(0/12\) versus \(8/12\) is \(p \approx 0.0013\), and the ordering is threshold-stable. Fit-only generalization is unchanged across rounds (\(0.704\) versus \(0.733\) at \(T=1024\)), reproducing the reliability-versus-quality separation.

The reading is clean: the lever is the commutativity-breaking coupling, not the block size. One residual caveat applies to the cross-family anchor only: rounds \(= 1\) is a pure rotation with no \(\tanh u\) scale channel while the \(2\)D composer carries one, so their equivalence is suggestive; the within-family monotonicity \(0/12 \to 6/12 \to 8/12\) under one factory and one protocol is the controlled result.

Composer reliability: Givens vs. delta, matched state and native state¶

The two composers were first separated using a lab reimplementation of the delta rule (a DeltaProduct-style two-reflection layer, not the packaged DeltaMinGRU), which on S3-hier fit \(6/6\) seeds at its native \(1024\)-element state against GivensMinGRU's \(8/12\) at \(64\) elements: an unmatched-state comparison that motivated, but did not settle, a state-controlled trial. Two later rounds close that gap with the packaged chunked-WY DeltaMinGRU, and their combined result is what promotes delta to the default recommendation when state is free to grow.

Matched-state trial, CPU (hetero-loop-20-pd64, hetero-loop-21-pd1024, 12 seeds each, torch 2.5.1). Before any seed launched, a delta-bridge selftest proved the packaged DeltaMinGRU state_dict-compatible with the lab's sequential mixer in both load directions, forward parity \(\leq 2.7 \times 10^{-6}\), so this round trains the packaged code path, not a stand-in. Same S3-hier, best-val@128 protocol, 1600-step budget:

At \(n=12\), neither matched-state contrast reaches significance (delta@64 vs givens@64: Fisher \(p = 0.2203\); delta@1024 vs givens@64: \(p = 0.0932\)), a power limitation, not a null result, as the next round shows. What the round does establish cleanly is cost and memory: delta at its native \(1024\)-element state fits every seed at \(\frac{1}{16}\) the per-step cost and \(0.57\times\) the memory of givens@64, and delta's own matched-state fit-only length generalization (\(0.721\)) sits within noise of givens' (\(0.733\)): reliability differs, quality of the fits that land does not.

GPU 36-seed campaign (hetero-gpu36-sg8, hetero-gpu36-pd64, hetero-gpu36-pd1024; torch 2.8.0+cu128, NVIDIA L4, triton 3.4.0, a separate stratum, never pooled with the CPU rows above), each mixer on its best practical GPU path (GivensMinGRU under the forced Triton scan, DeltaMinGRU under torch.compile):

At triple the seed count the two matched-state contrasts resolve: givens vs. delta@64 is now decisive (\(25/36\) vs \(10/36\), Fisher \(p = 0.00084\)), the CPU round's tie was underpowered, not an absence of effect, and the mechanisms fail differently: delta@64's non-fits sit at hard chance plateaus, while givens' non-fits are often near-misses. Delta at native state also beats givens@64 (\(35/36\) vs \(25/36\), \(p = 0.0030\)), and state size dominates within the delta family itself (\(10/36\) vs \(35/36\), \(p \approx 3.9\times10^{-10}\)). Per-seed wall time is parity across all three arms, \(\approx 15\)–\(18\)s: on this GPU stratum, GivensMinGRU's Triton scan erases the CPU-side \(16\times\) gap, and delta's torch.compile path is no cheaper than Givens' Triton path at these shapes. That is the honest GPU story: it is not a speed argument for either composer, it is a reliability and state-economics one.

Reading the two composers together, without overclaiming either way. DeltaMinGRU at its native state is the single most reliable composer measured, on both strata (\(12/12\) CPU, \(35/36\) GPU), at cost and memory that stay nearly flat as state grows: this is the promoted default when state is free to grow. GivensMinGRU is the decisively more reliable composer at a matched, small state budget (\(25/36\) vs \(10/36\) on the stratum with the power to show it), and its small-state fit cohort (like delta@64's) holds up better at extreme extrapolation length than the large-state delta cohort (\(0.679\) givens / \(0.699\) delta@64 vs \(0.570\) delta@1024 at \(16\times\) train length, fit-only, GPU stratum). Do not read delta@64's low full-seed accuracy means as "delta is less accurate": they are a fit-rate artifact. Seeds that fit generalize alike regardless of mechanism (matched-state fit-only \(0.721\) delta vs \(0.733\) givens at \(T=1024\), CPU), so delta@64's pooled mean is dragged down by seeds that plateau at chance, not by the mechanism generalizing worse when it does fit. Symmetrically, delta@1024 is the most accurate arm at or near training length in both strata (CPU: \(1.000\)@64, \(0.988\)@256; GPU: \(0.984\)@64, \(0.971\)@256) and only trails on the long-extrapolation tail. The two composers are not ranked; they answer different questions about your deployment. Full statistics, the pinned-CPU and GPU environment blocks, and the per-seed provenance are in experiments/EXPERIMENTS.md (rounds hetero-loop-20-pd64, hetero-loop-21-pd1024, hetero-gpu36-sg8/pd64/pd1024) and experiments/bench/gpu36_env.json.

The trade it does not escape¶

Orthogonality buys norm preservation: a product of Givens rotations has determinant \(+1\) and never shrinks or grows the state, so there is no amplitude decay across hundreds of compositions, unlike a diagonal scan whose eigenvalues in \((0,1)\) fade. But continuity means no attractor. Nothing pins a learned angle to the exact group element it approximates, so a small angle error compounds with sequence length exactly as an unsnapped RotationMinGRU's does, and accuracy still decays by \(T=1024\). The pooled decay from \(0.949\) at \(T=64\) to \(0.613\) at \(T=1024\) is this effect.

This is where the honest ceiling sits. Exactness at length remains unique to the snapped \(2\)D composer's rare exact seed, which reaches \(0.983\) at \(T=1024\) (the best length-generalization figure measured on S3-hier), and no continuous composer, Givens included, reaches it. RotationMinGRU's straight-through angle snapping manufactures an attractor at exact group angles that GivensMinGRU's continuous, exactly-orthogonal transition deliberately has none of. Snapping the Givens angles is the obvious hybrid and an open question, not a promise. The trade is therefore: GivensMinGRU fits far more reliably (8/12 versus a rare snapped winner) but tops out short of exact; the snapped \(2\)D composer almost never fits but is exact when it does.

DeltaMinGRU does not escape a version of the same trade, by a different route. Orthogonality is what gives Givens its norm preservation; the delta rule's memory matrix carries no such constraint, so nothing bounds its growth or decay across compositions either. The measured symptom is the same shape as Givens': fit-only length generalization degrades with \(T\) regardless of state size (matched-state CPU: \(0.939 \to 0.721\) from \(T{=}512\) to \(T{=}1024\); native-state GPU: \(0.843 \to 0.570\)), and the native-state cohort degrades more, not less, than the small-state one: more capacity buys fit reliability, not a longer-range attractor. Neither composer here manufactures the kind of exact attractor RotationMinGRU's angle snap does; both are reliability levers within the continuous-composer family, not length-generalization ones.

Time-aware decay on an orthogonal block¶

Every mixer optionally scales its transition by a per-event decay term \(\gamma = \exp(-\lambda\, f(\Delta t))\), with \(\lambda \geq 0\) per block and \(f\) the identity or \(\log(1+\cdot)\); see Reference: mixers for the keyword arguments and How-to: choose a mixer for when to enable it. For GivensMinGRU the semantics are identical to RotationMinGRU's and worth stating because the interaction with orthogonality is exactly what makes it clean: the scalar \(\gamma\) multiplies the whole block, \(\gamma M_t\), and because a scalar commutes with the orthogonal block action, the composed rotation is recovered unchanged from the decayed matrix: only amplitude fades, direction is untouched. The injection \(b_t\) is never decayed. Since \(\lambda \geq 0\) and \(\Delta t \geq 0\), \(\gamma \in (0, 1]\), so decay can only shrink. The \(\Delta t = 0 \Rightarrow \gamma = 1\) contract holds at every position including \(t=0\), which is what keeps chunked-versus-full equivalence exact. Two modes exist: "fixed" (a scalar buffer, not learned) and "learnable" (\(\lambda = \mathrm{softplus}(\rho)\), one \(\rho\) per block, initialized so \(\lambda\) equals decay_rate at construction). Note that GivensMinGRU's class-default decay_rate is \(1.0\), a strong default that is directly relevant to the backward-pass decision below.

The backward pass: why division-based reversal was rejected¶

The most consequential design decision in the whole Triton backend concerns how GivensMinGRU's (and RotationMinGRU's) angle-fused kernel computes gradients. The forward fast path (Kernel 4 in the scan-kernel design) carries the state vector in registers and applies the factored plane rotations directly from the angles, never materializing or scanning the \(k \times k\) transition matrices. That is the source of its memory win. The question is the backward: to accumulate gradients of the angles, the scale channel, the injection, the decay, and \(h_0\), the adjoint recurrence needs each intermediate state \(h_{t-1}\), and the forward did not save them.

There are two ways to get \(h_{t-1}\) back. The tempting one is reversal: since each forward step is \(h_t = \gamma_t\, R(\theta_t)\, S_t\, h_{t-1} + b_t\) with \(S_t\) the per-block scale, one can algebraically invert it (\(h_{t-1} = S_t^{-1} R(\theta_t)^\top (h_t - b_t) / \gamma_t\)) and walk the states backward from a checkpoint stored every \(C\) steps, reconstructing the interior of each chunk by division. This is attractive because it stores only every \(C\)-th state.

It is also unsound here, and the repository has the measurement to prove it. The per-block scale \(\mathrm{diag}(1, \tanh u_t)\) is near-singular whenever \(\tanh u_t \approx 0\), which is typical at initialization for the rotation variant, and dividing out a near-singular scale (or a strong decay \(\gamma\)) amplifies floating-point roundoff: reversal error grows as \(\sigma_{\min}^{-\text{chunklen}}\), where \(\sigma_{\min}\) is the smallest singular value of the per-step map. The committed emulation (experiments/reversal_emulation.py, artifact experiments/bench/reversal_emulation.md) isolates the division channel at checkpoint interval \(C = 64\) with per-step decays sampled in \([\gamma_{\min}, 1)\), fp32, \(T=4096\), and measures the decay-gate gradient error (max deviation over max gradient magnitude) that a reversal backward would feed the optimizer:

A stricter per-element relative metric reaches \(4 \times 10^{14}\) at \(\gamma_{\min}=0.23\), and the original design-phase emulation (a different normalization, run while the decision was being made) reached \(9.3 \times 10^{19}\) at the default-decay regime, the number that ended the debate. The exact mantissa is protocol-dependent; the conclusion is not: below mild decay the reconstructed gradient is not an approximation but noise amplified by tens of orders of magnitude. The pass/fail boundary sits near \(\gamma \approx 0.87\) at \(C = 64\); a decay of \(\gamma = 0.8\) already fails by roughly \(100\times\). Because GivensMinGRU's class-default decay_rate is \(1.0\), a user enabling decay would routinely land in the divergent regime, and under MINGRU_SCAN=auto the corruption would be silent: a wrong gradient, not a crash or a warning. That was deemed unacceptable.

The resolution (recorded in the module documentation of mingru.min_gru and mingru.triton_scans) rejected division-based reversal entirely for both rotation-family mixers, and removed the interval parameter so it cannot be re-enabled. The backward now uses an exact \(C = 1\) stored-state recompute: every \(h_{t-1}\) is read directly from the stored forward output, never reconstructed by inverting the forward step, so gradients are exact at any decay strength and any initialization. The elegance is that this costs zero extra memory: the forward output tensor is the module's return value, alive regardless, so it doubles as the anchor source with no extra allocation. The angle-fused path's memory win over the generic Phase-1 backward (never materializing the \(k \times k\) transitions) is untouched by the change. A chunk-buffered forward recompute (numerically stable, no inversion) remains a benchmark-justified future option if the per-step state re-reads ever prove to cost material wall-clock, but it is not needed today.

The memory win is measured. At the lab shape \(B=128\), \(T=64\), \(d=64\), \(k=8\), backward peak memory for the angle-fused \(C=1\) path is 38.25 MB against 395.09 MB for the generic Phase-1 backward that materializes the transitions, a \(10.3\times\) reduction (experiments/bench/scan_memory.md). The full kernel-design context, the fp32-accumulation story, and the measured speedup table live in the Triton scan kernels article.

Costs and the two-axis recommendation¶

Cost, memory, and reliability separate along two axes: per-token state budget (must it stay small, or can it grow) and deployment/extrapolation length (near training length, or many multiples out). Report the evidence by hardware stratum and never mix them without saying so: the CPU numbers above (pinned torch 2.5.1) and the GPU numbers above (torch 2.8.0+cu128, NVIDIA L4, triton 3.4.0) are not comparable to each other.

CPU: delta is the fast path by roughly \(16\times\), at any state size. GivensMinGRU's parallel \(k \times k\) scan is not the cheap path here: one uncontended forward+backward step at \(B=128\), \(T=64\) costs \(0.9493\)s against DeltaMinGRU's chunked-WY \(0.0577\)s, and the gap persists at \(T=1024\) (\(24.9996\)s against \(1.4541\)s). See "Flat cost and memory as state grows" above for how that gap holds as delta's state grows to \(4096\) elements while Givens' does not.

GPU: parity in speed, not in reliability. On the L4, per-seed training wall time is roughly the same across all three composer arms (\(15\)–\(18\)s): Givens' Triton scan path erases the CPU-side \(16\times\) gap, so speed is not what should drive a GPU choice between them. Delta's own GPU path is torch.compile, not a hand-written kernel: the CUDA fusion-headroom probe (experiments/bench/gpu_delta_probe.md) measured eager chunked-WY against an approximate matmul-FLOP floor and against torch.compile across five shapes, and compile recovers \(70\)–\(91\%\) of the available fusion headroom with zero kernel-engineering effort (residual \(1.22\times\)–\(1.80\times\) against a deliberately optimistic floor). Verdict from that round: a hand-written Triton chunked-WY kernel is not worth building today. torch.compile is the recommended CUDA path for DeltaMinGRU, and the question reopens only if a workload appears where the residual matters at scale.

The recommendation.

• State free to grow → DeltaMinGRU at native state (mixer="delta") is the promoted default. It is the most reliable composer measured on both strata (\(12/12\) CPU at state 1024, \(35/36\) GPU under torch.compile), at cost and memory that stay nearly flat as state grows. Main risk: its fit cohort's own long-extrapolation tail is the weakest of the reliably-trainable configurations measured (\(0.570\) at \(16\times\) train length, GPU fit-only). Pick it for near-length reliability, not extreme extrapolation.
• State must stay small (a matched \(64\)-element budget) → GivensMinGRU (mixer="givens", block_size=8, rounds=3) is the choice. It is decisively more reliable than delta at the same small state on the stratum with the power to show it (\(25/36\) vs \(10/36\), \(p = 0.00084\)), and on GPU it carries the Triton-accelerated scan that makes it cost-competitive at that state size. Main risk: like every continuous composer here it decays by \(T=1024\) and never reaches the snapped \(2\)D composer's rare exact seed.
• Extreme-length, fit-only extrapolation (\(16\times\) train length) favors the small-state composers. Both givens (\(0.679\)) and delta@64 (\(0.699\)) lead delta@1024's fit cohort (\(0.570\)) at that horizon: if a deployment's binding constraint is extrapolation length rather than fit reliability or per-token state, prefer whichever small-state composer's fit reliability is acceptable for the task, not delta's native size.
• If tokens are the operations directly (the plain S3 shape, no extraction stage), a single RotationMinGRU layer under the best-val@128 protocol is the more direct tool than either composer here. If the task only needs sign or parity dynamics, a diagonal SignedMinGRU is sufficient and cheaper than any of the non-commutative composers.

The full decision tree, evidentiary basis, and main risks per branch are in How-to: choose a mixer and experiments/TECHNICAL_REPORT.md §10 (predates this promotion; superseded by dated addenda pointing back here).

See also¶

• Reference: mixers: the GivensMinGRU and DeltaMinGRU constructors, keyword arguments, and validation rules.
• Reference: scan operations: matrix_affine_scan, the \(k \times k\) scan GivensMinGRU runs on.
• Triton scan kernels: the angle-fused Givens kernel, the fp32/TF32 story, and the measured speedups and memory result.
• Tutorials: two-layer stacks: building the signed-then-composer extract-then-compose stack.
• How-to: choose a mixer and How-to: reproduce the evidence: picking a composer and rerunning the S3-hier row.
• experiments/TECHNICAL_REPORT.md and experiments/EXPERIMENTS.md: the review-hardened report and the round-by-round ledger with the full multi-seed tables, the rounds ablation, and the delta-vs-Givens matched-state and GPU campaigns.