AI resilience

Capacity of AI systems to sustain safe, reliable, and intended operation under internal faults, adversarial interference, distributional shifts, and infrastructure disruptions.

AI resilience denotes the combination of algorithmic robustness, system-level fault tolerance, detection-and-recovery mechanisms, and operational practices that together ensure AI services continue to meet performance and safety requirements when exposed to perturbations. At the model level this encompasses adversarial robustness, certified guarantees, out-of-distribution detection and domain adaptation; at the training and learning level it includes techniques from continual learning, regularization, and robust optimization; at the systems level it requires redundancy, monitoring, graceful degradation, rollback and provenance-aware deployment workflows. Effective AI resilience bridges ML (Machine Learning) research (robustness metrics, certification methods, uncertainty quantification) with software-engineering constructs (failover, observability, testing under realistic distributional shifts) and governance (incident response, model cards, safety envelopes), and is evaluated using adversarial/shifted benchmarks, stress tests, and operational SLAs tailored to the application domain.

First recorded uses of the exact phrase trace to mid-2010s in research and engineering contexts (≈2014–2016), with the term gaining wide popularity and mainstream attention from about 2018–2023 as adversarial ML, robustness, and deployment failures in critical systems highlighted the need for end-to-end resilient AI.