Optimal Transport Theory

Optimal transport theory is a mathematical framework concerned with finding the most efficient way to move mass or probability from one distribution to another, minimizing a defined cost function in the process. In machine learning, this translates to a principled method for comparing probability distributions that respects the underlying geometry of the data space. The central quantity of interest is the Wasserstein distance (also called the Earth Mover's Distance), which measures how much "work" is required to reshape one distribution into another — where work is the product of mass moved and the distance it travels. Unlike simpler divergences such as KL divergence, Wasserstein distances remain meaningful even when two distributions have non-overlapping support, making them far more robust in practice.

The computational machinery behind optimal transport involves solving a linear program over the space of joint distributions (called transport plans) whose marginals match the source and target distributions. The classical Monge formulation seeks a deterministic mapping, while the Kantorovich relaxation allows probabilistic couplings, making the problem tractable via linear programming. A major practical breakthrough came with the introduction of the Sinkhorn algorithm and entropic regularization by Marco Cuturi in 2013, which made approximate optimal transport computations fast enough to embed directly into neural network training loops via automatic differentiation.

Optimal transport has since become a versatile tool across many areas of machine learning. In generative modeling, Wasserstein GANs use the Wasserstein distance as a training objective, producing more stable training dynamics and better sample quality than earlier GAN formulations. In domain adaptation, optimal transport provides a natural way to align feature distributions across source and target domains. It also appears in point cloud comparison, shape interpolation, fairness-aware learning, and the analysis of neural network loss landscapes. The theory connects geometry, probability, and optimization in a way that gives practitioners both interpretable metrics and actionable transport maps.

The growing availability of scalable solvers — including GPU-accelerated implementations in libraries like POT (Python Optimal Transport) and built-in support in JAX and PyTorch ecosystems — has made optimal transport increasingly accessible. What was once a niche topic in pure mathematics is now a standard component of the modern ML toolkit, valued for its geometric intuition and its ability to handle distributional comparison tasks that simpler divergences handle poorly.