minGRU · Benchmark validation on public tasks

How to read this

A fit is trainability at a fixed bar — measured, then anchored.

Every task's fit bar was frozen before the matrices ran. A cell counts how many seeds cleared their own validation-metric bar, not how well they generalize; the two reads are always reported apart.

Nine arms. The six packaged single-stack mixers (log, signed, rotation, signed-rotation, givens, delta), the two promoted heterogeneous stacks (signed-givens, signed-delta), and a depth-matched classical gru control (2-layer nn.GRU, d_model 64).
One stratum, disclosed. Every number is the L4 stratum — device=cuda, torch=2.8.0+cu128, triton=3.4.0, scan=triton. It is never compared against the pinned-CPU rounds or the A100 kernel probes.
Generalization is separate from fit. Raw accuracy pools all seeds; fit-only conditions on the seeds that fit. Both travel together, so a low pooled mean is never silently read as "less capable."

val128 ≥ 0.99

36 seeds · symmetric-group word problems

MQAR

val_qacc ≥ 0.99

36 seeds · multi-query associative recall

psMNIST

val_acc ≥ 0.90

12 seeds · permuted-pixel MNIST

pendulum

val_mse ≤ 0.0014

36 seeds · irregular-time regression

arm	S5 (n=36)	MQAR (n=36)	psMNIST (n=12)	pendulum (n=36)
log	0/36	0/36	0/12	36/36
signed	0/36	0/36	0/12	36/36
rotation	0/36	0/36	0/12	36/36
signed-rotation	0/36	0/36	0/12	36/36
givens	0/36	0/36	0/12	36/36
delta	0/36	36/36	10/12	36/36
signed-givens	1/36	0/36	0/12	36/36
signed-delta	0/36	36/36	12/12	36/36
grucontrol	0/36	0/36	3/12	36/36

Per-task detail

Each task isolates a different mechanism.

Fisher-exact contrasts are two-sided vs the log reference. Generalization columns are word-problem / query / test accuracy, raw over fit-only.

MQAR · associative recall

val_qacc ≥ 0.99

A pure delta dissociation: only the delta family fits, every non-delta arm sits at zero.

36/36 delta and signed-delta both fit
0/36 all seven other arms, including gru
0.931 → 0.493 delta raw q-acc, 16 → 32 key-value pairs
0.928 → 0.690 signed-delta, same sweep

Fisher vs log: p = 2.32 × 10⁻¹³ · matches the Zoology recall-capacity tradeoff (arXiv:2312.04927)

psMNIST · accumulation ordering

val_acc ≥ 0.90

The delta family leads on fit rate; signed-delta is the only threshold-stable fitting arm.

12/12 signed-delta (best) · raw test 0.924
10/12 delta · raw test 0.905
3/12 gru control · raw test 0.885
0.290 givens (weakest region: stacked pure rotation)

Fisher vs log: signed-delta p = 7.40 × 10⁻⁷, delta p = 6.73 × 10⁻⁵

S5 · group composition

val128 ≥ 0.99

Solved at the matched config by exactly one arm — a single seed that generalizes cleanly.

1/36 signed-givens (continuous coupled-8D rotation + sign)
1.000 / 0.976 / 0.817 its fit-only acc @ T256 / 512 / 1024
0/36 every other matched arm, threshold-stable

Fisher vs log: p = 1 — a single-seed margin does not separate at n = 36. See the probe below.

pendulum · positive control

val_mse ≤ 0.0014

All nine arms fit at every robustness threshold — the harness trains end-to-end.

36/36 every arm, including norm-preserving mixers and gru
p = 1 Fisher vs log for all arms

The decay channel is not the discriminating axis here — this arm proves the pipeline, nothing more. A decay-isolating variant is a deferred follow-up.

Full per-task tables

The complete nine-arm matrices, transcribed verbatim from experiments/bench/. Generalization is raw / fit-only (n/a where no seed fit); fitting arms are highlighted, the gru control in amber.

S5 · symmetric-group word problems · val128 ≥ 0.99 · n = 36 · bench_s5.md

arm	seeds	fits	acc@256	acc@512	acc@1024	params
log	36/36	0/36	0.016 / n/a	0.012 / n/a	0.010 / n/a	98,936
signed	36/36	0/36	0.025 / n/a	0.019 / n/a	0.015 / n/a	107,256
rotation	36/36	0/36	0.020 / n/a	0.015 / n/a	0.012 / n/a	107,384
signed-rotation	36/36	0/36	0.016 / n/a	0.013 / n/a	0.011 / n/a	107,320
givens	36/36	0/36	0.015 / n/a	0.012 / n/a	0.010 / n/a	111,544
delta	36/36	0/36	0.016 / n/a	0.012 / n/a	0.011 / n/a	133,256
signed-givens	36/36	1/36	0.058 / 1.000	0.049 / 0.976	0.038 / 0.817	109,400
signed-delta	36/36	0/36	0.018 / n/a	0.014 / n/a	0.011 / n/a	120,256
gru	36/36	0/36	0.024 / n/a	0.019 / n/a	0.017 / n/a	65,400

MQAR · multi-query associative recall · val_qacc ≥ 0.99 · n = 36 · bench_mqar.md

arm	seeds	fits	acc@256, 16 pairs	acc@256, 32 pairs	params
log	36/36	0/36	0.112 / n/a	0.081 / n/a	91,712
signed	36/36	0/36	0.044 / n/a	0.036 / n/a	100,032
rotation	36/36	0/36	0.083 / n/a	0.064 / n/a	100,160
signed-rotation	36/36	0/36	0.064 / n/a	0.051 / n/a	100,096
givens	36/36	0/36	0.030 / n/a	0.030 / n/a	104,320
delta	36/36	36/36	0.931 / 0.931	0.493 / 0.493	126,032
signed-givens	36/36	0/36	0.036 / n/a	0.034 / n/a	102,176
signed-delta	36/36	36/36	0.928 / 0.928	0.690 / 0.690	113,032
gru	36/36	0/36	0.111 / n/a	0.076 / n/a	58,176

psMNIST · permuted-pixel MNIST · val_acc ≥ 0.90 · n = 12 · bench_psmnist.md

arm	seeds	fits	acc@test	params
log	12/12	0/12	0.784 / n/a	84,234
signed	12/12	0/12	0.857 / n/a	92,554
rotation	12/12	0/12	0.571 / n/a	92,682
signed-rotation	12/12	0/12	0.736 / n/a	92,618
givens	12/12	0/12	0.290 / n/a	96,842
delta	12/12	10/12	0.905 / 0.908	118,554
signed-givens	12/12	0/12	0.651 / n/a	94,698
signed-delta	12/12	12/12	0.924 / 0.924	105,554
gru	12/12	3/12	0.885 / 0.897	38,474

psMNIST block-order ablation · same arm, same task, same protocol · only the composer block order differs

block order	round	fits	acc@test	params
rotation-first ["rotation","signed"] · superseded	bench-psmnist-02	0/12	0.868 / n/a	92,618
signed-first ["signed","rotation"] · current	bench-psmnist-rotfix-01	0/12	0.736 / n/a	92,618

The corrected extract-then-compose order costs 0.868 → 0.736 raw test accuracy (−0.132), with neither order clearing the 0.90 bar. psMNIST is accumulation-ordering, not group composition: the block that touches the raw pixel stream first shapes the accumulation, and the richer non-diagonal rotation block on the input (rotation-first) accumulated better than routing the stream through the diagonal sign block first. The original 0.868 was partly an artifact of that accumulation-favorable rotation-first order; the apples-to-apples number, with the arm in the same architecture as its signed-givens/signed-delta siblings, is 0.736. Block order is task-dependent: extract-then-compose helps group composition and hurts accumulation.

pendulum · irregular-timestep regression · val_mse ≤ 0.0014 · n = 36 · bench_pendulum.md

arm	seeds	fits	params
log	36/36	36/36	83,970
signed	36/36	36/36	92,290
rotation	36/36	36/36	92,354
signed-rotation	36/36	36/36	92,226
givens	36/36	36/36	96,466
delta	36/36	36/36	118,162
signed-givens	36/36	36/36	94,306
signed-delta	36/36	36/36	105,162
gru	36/36	36/36	38,338

S5 design-correction probe

Two arms were handicapped by config — one recovers, one doesn't.

A descriptive probe (no Fisher contrast) that separates a genuine mechanism limit from an experiment-design artifact, by lifting the two under-powered S5 configs.

The Householder threshold

Raising the delta product count recovers S5; adding a snap order does not rescue rotation.

A k-cycle needs k−1 reflections, and S5's 5-cycle needs four — so nh = 4 is the threshold. signed-delta at nh = 2 (the matched config) was under-powered by design: at nh = 3 it is still 0/36, but at nh = 4 it reaches 7/36, with clean fit-only generalization (0.992 / 0.892 / 0.618 at T256 / 512 / 1024).

Adding S5's order-5 to the rotation snap grid (signed-rotation-k5, snap = (2,3,4,5,6)) leaves it at 0/36. This probe now also runs the corrected signed-first block order, so the rotation family's matched zero is confirmed a real mechanism limit: not a missing snap order, and not the earlier rotation-first block-order confound. Net across matched and probe evidence, S5 has two solving arms: continuous signed-givens (1/36) and Householder signed-delta-nh4 (7/36).

S5 design-correction probe · descriptive population, no Fisher contrast · bench_s5_probe.md

arm	seeds	fits	acc@256	acc@512	acc@1024	params
signed-rotation-k5	36/36	0/36	0.015 / n/a	0.012 / n/a	0.011 / n/a	107,320
signed-delta-nh3	36/36	0/36	0.030 / n/a	0.020 / n/a	0.014 / n/a	128,836
signed-delta-nh4	36/36	7/36	0.247 / 0.992	0.211 / 0.892	0.144 / 0.618	137,416

arm	seeds	fits	acc@test	params
gru-large	12/12	12/12	0.922 / 0.922	596,234

Cross-task reading

Two dials for two task regimes.

Not a single winner, and not "delta beats givens everywhere." The read is which mechanism each task structure rewards.

Delta — the broad workhorse

Fits across three task structures once its dials are set right.

Associative recall (MQAR) — only the delta family fits.
Accumulation ordering (psMNIST) — signed-delta 12/12, delta 10/12 lead the fit rate.
Group composition (S5) — recovered once nh is large enough (signed-delta-nh4, 7/36).

dial · state sizedial · nh (Householder count)

Givens — a narrow specialist

One thing, done at a small fixed state.

Group composition is its lane — the continuous coupled-8D rotation + sign stack (signed-givens) is the only matched S5 fit.
It does not lead recall or accumulation; adding a sign channel to givens even hurts accumulation on psMNIST.
The earlier matched-small-state S3-hier win still stands — that finding is task-specific and separate.

What this round does not claim

Three guardrails held on purpose.

Not "delta beats givens everywhere." Two dials for two regimes; the matched-small-state Givens result on S3-hier stands. Not evidence a GRU cannot state-track. The 0/36 on S5 is a same-budget outcome, not a capability limit (Illusion of State, arXiv:2404.08819). No cross-stratum comparison. The L4 numbers here are never mixed with the pinned-CPU or A100 rounds.

Validation, not a leaderboard.