VICReg (Variance-Invariance-Covariance Regularization)

VICReg (Variance-Invariance-Covariance Regularization) is a self-supervised learning objective introduced by Bardes, Ponce, and LeCun (Meta FAIR, May 2021) that prevents representational collapse through three explicit regularization terms applied to embedding-space outputs: a variance term that maintains the standard deviation of each embedding dimension above a threshold (preventing all embeddings from collapsing to a constant); an invariance term that minimizes the distance between embeddings of augmented views of the same image (the standard SSL prediction target); and a covariance term that decorrelates the embedding dimensions (preventing collapse into a lower-dimensional subspace).

The key innovation is that VICReg does not require negative samples or contrastive pairs — the covariance regularization term alone is sufficient to prevent collapse. This makes it computationally cheaper than contrastive methods (SimCLR, MoCo) while achieving comparable or better downstream performance. The architecture uses a Siamese-style encoder with two branches processing augmented views, but unlike contrastive methods, only the invariance term requires positive pairs.

VICReg has become a foundational SSL building block: it underpins Barlow Twins' redundancy-reduction formulation, has been applied to audio and sequential data (Barlow Twins extensions), and was used by the LeVJEPA paper (Kuhn et al., Aug 2026) to demonstrate that the standard JEPA heuristics — target encoder, masked prediction, stop-gradient — can be replaced with a single regularization-based objective. VICReg is in the same family as barlow-twins, byol, simclr, and w-mse (Whitening MSE), all of which address the collapse problem in self-supervised learning through different mechanisms.