---
title: Looped Transformer
type: vocabulary
url: "https://www.envisioning.com/vocab/looped-transformer"
summary: Transformer architecture that reuses the same block weights multiple times to add depth without adding parameters.
year: 2024
generality: 0.55
---

# Looped Transformer

Transformer architecture that reuses the same block weights multiple times to add depth without adding parameters.
A looped transformer applies the same stack of layers to the same hidden representation more than once, sharing weights across iterations. Instead of stacking N distinct transformer blocks for depth, the model unrolls a smaller block K times. When the loop count is fixed, the design is sometimes called recurrent depth; when the count can vary per token, looped transformer.

The design builds on universal transformers, deep equilibrium models, and weight-tied recurrent networks. In 2024 and 2025, it was used to scale depth on a fixed inference budget. According to The Information, OpenAI's Astra model, reported in September 2026, uses a recurrent depth or looped transformer design.

Because weights are shared across iterations, effective compute grows with the loop count while memory does not. Mixture-of-Recursions adds a learned router that assigns each token a per-input loop budget, letting the model allocate depth where it is needed. Looped transformers compose with grouped-query attention, RoPE, and RMSNorm, which is why open-weights models like Nanbeige 4.2 (3B) ship as looped architectures.

---
Source: Envisioning — Technology Research Institute (https://www.envisioning.com/vocab/looped-transformer)