Thinking Without Talking - Recurrent Depth
2026-09-02
By Zhenzhao Tu
The Timeline of Recurrent Depth
-
February 2025: Researchers (Jonas Geiping and team) drop the foundational paper proving you can scale test-time compute by reasoning implicitly in latent space.
-
October 2025: ByteDance releases Ouro. They prove a 2.6B model using recurrent loops can match the reasoning power of massive 8B models without just memorizing facts.
-
September 2026: OpenAI adopts this architecture for Astra, officially cementing latent reasoning as the new frontier of AI scaling.
The Core Concept: Trading the Whiteboard for the Mind
Think about how you solve a math problem. Traditional AI uses visible "Chain of Thought" (CoT)—it writes every single step down on a whiteboard, generating text tokens one by one. Recurrent Depth forces the model to put down the marker. Instead of a chain-of-tokens, it uses a continuous "chain-of-vectors," processing the logic entirely in its hidden, latent space before it ever speaks a single word.
[Interactive figure: Recurrent Block Information Flow]
Step through a simplified Transformer-style recurrent block and compare one-round processing with repeated internal passes.
The Architecture: The Nested Engine
A standard transformer works in three phases: a Prelude (turning text into vectors), a Core (processing), and a Coda (turning vectors back to text).
In a classical model, that Core is a massive skyscraper of 100 physically distinct layers. Data moves up the stack once. Recurrent Depth demolishes the skyscraper. It replaces those 100 linear layers with a single Recurrent Block. The data enters this block and loops around inside it like a washing machine, re-using the exact same parameters over and over. And if you zoom in? The machinery inside that looping block is exactly the same standard transformer engine we’ve always used: Multi-Head Attention and a Feed-Forward Network.
How Latent Reasoning Actually Works (The Math)
If you want to solve a hard problem using traditional CoT, you literally write down Option A, evaluate it, write down Option B, and so on.
Recurrent Depth doesn't list anything. Because its mathematical representations (embeddings) are massively high-dimensional, the vector holds every potential solution in superposition simultaneously. The recurrent block acts as a mathematical filter inside a while-loop. Every time the data loops, the attention mechanism shrinks the noise of the bad options and amplifies the coordinates of the correct one.
It is constantly approximating toward a critical point. Over and over, it filters the superposed noise until it converges onto the exact, low-dimensional manifold of the true solution. A simple question converges to that critical point in two loops. A complex question might loop fifty times to capture the right structure.
The Catch: Shortcut Learning and Local Minima
This architecture is incredibly efficient for structured logic, but it hits a wall with open-ended complexity.
Because everything is a numerical approximation converging to a critical point, the optimization landscape can trick the model. Imagine a bumpy mathematical landscape with multiple valleys. If the algorithm spots an "obvious but slightly wrong" answer, it rolls into that shallow valley (a local minimum) and gets trapped, completely missing the deeper, true solution (the global minimum).
[Interactive figure: Shallow and Deep Minima]
A ball descends across a flat loss map with one shallow local minimum and one deeper global minimum. Adjust its starting position and step size, then use Step or Play to see which basin captures it.
Worse, because the model isn't writing words down on a whiteboard, it has no "save points." It cannot backtrack. If it loops too many times on a messy problem, the math over-smooths, the nuances collapse, and it confidently spits out a lazy shortcut. For highly divergent, ambiguous problems, we still need visible text to keep competing ideas safely separated.
Comments