Minimax Loss
An optimization strategy that minimizes the worst-case maximum loss an adversary can cause.
Minimax loss is an optimization objective rooted in game theory that seeks to minimize the maximum possible loss an adversary could inflict on a model. Rather than optimizing for average-case performance, minimax loss treats the training or evaluation process as a two-player zero-sum game: one player (the model) attempts to minimize loss, while an opposing player (an adversary or nature) attempts to maximize it. The resulting solution is a strategy that performs as well as possible even under the most unfavorable conditions, making it especially valuable when reliability under worst-case scenarios is critical.
In practice, minimax loss appears prominently in adversarial machine learning, where models must remain accurate despite deliberately crafted inputs designed to cause failures. Adversarial training, for example, incorporates minimax loss by alternating between generating worst-case perturbations and updating model parameters to reduce their effect. This framework also underpins Generative Adversarial Networks (GANs), where a generator minimizes loss while a discriminator maximizes it, producing a minimax game whose equilibrium yields a generator capable of producing realistic synthetic data.
The mathematical foundation of minimax optimization traces back to John von Neumann's minimax theorem from the 1920s, but its direct application to machine learning accelerated significantly with the formalization of adversarial robustness research in the 2010s. Ian Goodfellow's 2014 introduction of GANs brought minimax loss into mainstream deep learning discourse, while concurrent work on certified robustness and adversarial training by researchers like Madry et al. established minimax formulations as a principled approach to building robust neural networks.
Minimax loss matters because real-world deployments of ML systems frequently encounter distribution shift, noisy inputs, or deliberate manipulation. Models trained under standard empirical risk minimization can be brittle, failing catastrophically on edge cases. By explicitly accounting for worst-case scenarios during training, minimax loss encourages models that generalize more robustly and are harder to exploit, making it a foundational concept in safety-critical and security-sensitive applications of machine learning.
