Latent Reasoning

Title: Latent reasoning Slug: latent-reasoning

Latent reasoning is the practice of performing multi-step reasoning inside a neural network's hidden states rather than expressing intermediate steps as visible chain-of-thought tokens in the output. A model doing latent reasoning manipulates its residual stream, attention patterns, or iteratively-looped layer activations to work through a problem, and only the final answer emerges in the visible output. The reasoning itself is real: the model iterates over intermediate states, but the process never leaves the latent space in a form humans can read.

The technique sits at one end of a spectrum. That spectrum runs from fully externalized reasoning (classical chain-of-thought, where every intermediate step is a token) through partially externalized reasoning (CoT where the model is rewarded for short or hidden reasoning, or where intermediate scratchpads are discarded after use) to fully latent reasoning (recurrent depth, looped transformers, and similar approaches where the entire reasoning trace lives in hidden states). The shift toward latent reasoning is driven by inference economics. Externalized reasoning costs context window, costs tokens, costs KV-cache memory, and is bound by the model's ability to verbalize steps in a language it has been trained to produce. Latent reasoning frees the model from these constraints and lets it spend additional compute per token without paying the tokenization tax.

The technique draws on a long lineage of work that questions whether reasoning must be verbal to count as reasoning. Earlier instantiations include looped transformers, universal Transformers, and various forms of iterative refinement where a model's output is fed back into itself for additional computation. The 2026 wave is different because latent reasoning is the explicit design goal of a major deployed model, Astra, released in late summer 2026, with its recurrent-depth architecture, rather than an emergent property of a particular training run. Framing it as a deliberate architectural choice, with associated safety and interpretability consequences, marks the move of latent reasoning from a research curiosity to a production technique.

The trade-offs versus chain-of-thought reasoning are sharp. CoT offers a partial observability guarantee: a model's intermediate reasoning can be inspected, audited, or used as a forensic trace, a property that proved decisive during the investigation of the July 2025 Hugging Face supply-chain attack, where the visible reasoning of certain agentic systems let researchers reconstruct how malicious code had been triggered. Latent reasoning surrenders this observability in exchange for a more compact context footprint and the ability to iterate without paying token costs. A 2024 AI-safety paper co-authored by researchers from OpenAI, Anthropic, and Google DeepMind warned that latent reasoning models might not need to verbalize any of their thoughts and would thus lose the safety advantages that chain-of-thought confers, framing the trade-off as a central concern for the field.

The open questions for latent reasoning are largely about verification and control. If reasoning is hidden, how do you audit whether a model's decision was sound? How do you detect when a model's latent state encodes a step that, if verbalized, would reveal a policy violation or a deceptive strategy? How do researchers compare reasoning strategies across models when the strategies themselves are no longer observable? Several research directions, including interpretability of latent states, mechanistic analysis of looped computation, and behavioral evaluation rather than verbal-trace evaluation, are gaining urgency because latent reasoning makes the older chain-of-thought inspection toolkit less applicable.