When Reasoning Becomes a Dynamical System: A Look at Equilibrium Reasoners

Introduction

A persistent puzzle in modern machine learning is why some models improve dramatically when given more compute at inference time, while others plateau or degrade. Chain-of-thought prompting, search-based game agents, and iterative latent updates all spend extra inference cycles in pursuit of better answers, yet the internal mechanism that separates productive scaling from wasted compute has remained poorly characterized. The paper Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning by Benhao Huang, Zhengyang Geng, and Zico Kolter (CMU) offers a clean and falsifiable hypothesis: generalizable reasoning emerges when a learned latent dynamical system develops task-conditioned attractors, stable fixed points whose locations correspond to valid solutions.

The headline result is hard to ignore. On Sudoku-Extreme, a feedforward model reaches only 2.6% exact accuracy, while the Equilibrium Reasoner (EqR), trained with a cap of 16 iterations, generalizes to over 1,024 iterations at test time, an equivalent of roughly 40,000 unrolled layers, and exceeds 99% accuracy. What follows is an attempt to unpack why this works and what it implies for the broader study of reasoning.

From Feedforward Layers to Fixed-Point Dynamics

The first conceptual move in the paper is to treat repeated application of a learned module as a dynamical system rather than a deep network. Iterative reasoning models such as HRM and TRM already apply a weight-tied module repeatedly to update a latent state. EqR reframes this as a fixed-point system:

$$z^\star = f_\theta(z^\star; x)$$

where x is the task input and z is the latent state. This is a deliberate echo of the Deep Equilibrium (DEQ) perspective from Bai et al. (2019), but the authors push the framing further. The interesting question is no longer simply whether a state converges, but what landscape of attractors the learned dynamics induce: which fixed points exist, whether they are reachable from plausible initializations, and whether they align with the task metric.

The empirical case for weight tying is stark. Table 1 in the paper shows that scaling a feedforward model from 4 to 64 layers barely moves Sudoku accuracy (1.8% to 2.6%), and 256 layers run out of memory entirely. Maze accuracy stays at exactly 0.0% for all feedforward depths. Meanwhile, weight-tied iterative models climb steadily: TRM reaches 84.8% on Sudoku and 44.9% on Maze, and EqR reaches 99.8% and 93.0% respectively. The lesson is that depth alone is not the resource that matters. Reusing the same map repeatedly is what allows the system to behave like a contraction toward a solution rather than a fixed-length feedforward computation.

The conceptual picture the authors paint, summarized in their landscape figure, involves two surfaces. The lower surface is the task-metric landscape over candidate solutions, with a low-error basin around correct answers. The upper surface is the model's internal landscape over latent states, shaped by training. Learning f_theta amounts to aligning the basins of these two landscapes so that descending the internal landscape through iterative updates also moves toward a correct solution. This is an elegant reframing of training: it is not memorizing input-output pairs but constructing a differentiable surrogate whose attractors coincide with valid answers.

Convergence as a Diagnostic, and Two Axes of Scaling

The most useful contribution, in my view, is not the accuracy number but the diagnostic the framing provides. EqR measures convergence via the fixed-point residual:

$$|f_\theta(z; x) - z|$$

A lower residual means the latent state has settled closer to a fixed point. The central empirical finding is that this residual tightly tracks prediction error. Trajectories that converge better decode to lower-error answers. This is significant because it gives a task-agnostic, verifier-free signal of solution quality. Most test-time scaling methods rely on external verifiers, reward models, or task-specific priors to decide when an answer is good. EqR instead reads convergence directly off the dynamics.

From this diagnostic, the authors derive two explicit inference axes:

• Depth (D): the number of unrolled steps per trajectory. More iterations drive the residual down and let trajectories settle into attractors.
• Breadth (B): the number of stochastic trajectories launched from independent initializations, then aggregated.

The interaction between the two is the part worth dwelling on. The paper reports that breadth becomes effective only after sufficient depth. This makes intuitive sense within the attractor picture: launching many trajectories is only useful once each trajectory has enough iterations to meaningfully explore the landscape and settle into a basin. Restarting from many points without enough depth simply samples noise. With enough depth, multiple restarts cover different basins of attraction, some favorable and some spurious, and aggregation selects the favorable ones.

To make favorable attractors easier to reach, the authors introduce two lightweight training interventions: randomized state initialization and path stochasticity via noise injection. Both are recognizable techniques from the broader literature on robust optimization and stochastic dynamics, but here they serve a specific purpose: widening the basins of correct attractors so that more initial conditions flow into them.

My Take: The Missing Piece Is Basin Geometry

The attractor framing is compelling, but it currently describes convergence in aggregate. The natural next step, which I raised briefly when this paper crossed my feed, is to measure the geometry of the attractors directly rather than inferring it from residual statistics.

Concretely, at a converged state z*, the local behavior of the dynamics is governed by the Jacobian of f_theta evaluated at the fixed point. The eigenvalue spectrum of that Jacobian tells you the contraction rate along each direction. A fixed point with all eigenvalues well inside the unit circle is a strongly attracting solution that converges fast. A fixed point with eigenvalues near the boundary contracts slowly, and directions with magnitude near or above one indicate saddle-like behavior where trajectories linger before escaping.

If this picture holds, several predictions follow:

• Easy instances should converge to attractors with fast contraction and require only a handful of iterations, consistent with the paper's observation that simple cases settle in 1 to 5 steps.
• Hard instances should sit near saddle points or weakly contracting fixed points, explaining why they benefit from massive depth scaling.
• The slowest eigenvalue at convergence, or even an early-iteration estimate of the local spectrum, could serve as a per-instance difficulty predictor before committing to full inference.

This matters because the current method, as impressive as 40,000 effective layers sounds, almost certainly wastes enormous compute on instances that converge in five steps. An adaptive controller that reads the local contraction rate could allocate depth proportionally, stopping early on easy cases and reserving deep unrolling and broad restarts for genuinely hard ones. The fixed-point residual already provides a coarse stopping signal; a spectral estimate would provide a predictive one.

There is a tension worth flagging. DEQ-style implicit differentiation gives access to Jacobian-vector products relatively cheaply, so spectral probing is not free but is feasible. The open question is whether the spectrum at early iterations is informative enough to predict eventual difficulty, or whether it only becomes reliable near convergence, at which point the prediction is less useful.

Limitations and Open Questions

Several caveats temper the enthusiasm. The benchmarks here are controlled, structured tasks (Sudoku-Extreme, Maze-Unique) chosen precisely because memorization can be separated from generalization. Whether the attractor framing transfers to open-ended natural language reasoning, where there is no clean task-metric landscape and no unique fixed point, is entirely unestablished. The notion of a "correct attractor" presumes a well-defined solution structure that language tasks often lack.

There is also the question of spurious attractors. The paper acknowledges that failures correspond to spurious fixed points, and that scaling works by guiding trajectories into favorable basins. But nothing guarantees that the favorable basins are larger or more reachable than spurious ones. The training interventions help, yet a principled account of why training reliably produces a benign attractor landscape, rather than one riddled with confident wrong answers, is still missing.

Conclusion

Equilibrium Reasoners reframe test-time scaling as convergence toward learned attractors, and in doing so they provide something rare: a mechanistic, verifier-free diagnostic that predicts when more compute will help. The depth-breadth interaction and the tight coupling between fixed-point residual and accuracy are clean, testable claims. The most promising direction forward is to move from measuring convergence to measuring basin geometry, using the Jacobian spectrum at or near fixed points to predict difficulty and allocate compute adaptively. If that works, the field gains not just scalable reasoning but efficient scalable reasoning, where 40,000 layers are spent only on the problems that actually need them.