Large Reconstruction Model

A Large Reconstruction Model (LRM) is a transformer trained to predict a 3D representation of an object or scene from one or a few 2D images in a single forward pass, without the per-scene optimization that earlier methods relied on. Hong et al. introduced the term in 2023 (arXiv:2311.04400, ICLR 2024 Oral), training a roughly 500-million-parameter encoder-decoder transformer to regress a neural radiance field, in the form of a triplane, directly from one image. Prior single-image or few-image 3D methods were typically trained on small, category-specific datasets like ShapeNet. LRM instead trained end to end on around a million objects drawn from Objaverse and MVImgNet, producing a model that generalizes to real-world photos and images generated by other models.

The approach treats 3D reconstruction as a problem of data and scale. Instead of fitting a NeRF or mesh to a specific object over many gradient steps, the network amortizes that cost into a single pretrained model that runs in seconds. This made LRM one of the first demonstrations that the transformer scaling recipe used for language and image models also transfers to 3D. Follow-up systems extended the idea to multi-view inputs, mesh and Gaussian-splat outputs, and multi-object scenes, collectively described as feed-forward or amortized 3D reconstruction.

A persistent limitation is fine detail. Feed-forward models trade some texture and geometric fidelity for speed, historically lagging dense multi-view optimization on sharp textures. Later work has targeted this gap directly. LSRM (Li et al., 2026, arXiv:2604.05182) scales the transformer's context window with a sparse-attention mechanism so the model can process far more image and object tokens, narrowing the fidelity gap with dense-view optimization while remaining a single feed-forward pass.