JEPA (Joint Embedding Predictive Architecture)

Joint Embedding Predictive Architecture (JEPA) is a self-supervised learning framework in which a model learns by predicting abstract representations of data rather than reconstructing raw inputs. Proposed by Yann LeCun as a cornerstone of his vision for human-level AI, JEPA encodes two related views or segments of an input, such as different patches of an image or different time steps in a sequence, into a shared embedding space. A predictor network then learns to map one encoded representation to another, with prediction happening entirely in latent space rather than in pixel or token space.

This design choice has specific consequences. Generative models that reconstruct raw inputs must account for every irrelevant detail, including the exact texture of a surface or the precise color of a background, and this requirement can interfere with learning semantically meaningful structure. By predicting in embedding space, JEPA avoids this problem. The model can discard low-level noise and concentrate on higher-level patterns that carry predictive information. A stop-gradient or exponential moving average target encoder, similar to techniques used in BYOL and DINO, is typically used to stabilize training and prevent representational collapse.

Image-JEPA (I-JEPA), introduced by Meta AI in 2023, showed that this approach could learn strong visual representations without relying on hand-crafted data augmentations, and it outperformed many contrastive and generative baselines on downstream tasks. Video-JEPA (V-JEPA) extended the framework to temporal prediction across video frames. JEPA also represents a departure from the dominant paradigm of large generative models. Where those models learn to produce outputs, JEPA learns to predict the world's structure in a compact, abstract form. LeCun argues that this property is essential for building efficient, generalizable world models that can support planning and reasoning.