Nested learning describes architectures and training procedures where learning itself is structured into nested optimization levels—commonly an inner loop that performs rapid adaptation (weights, fast parameters, or per-task updates) and an outer loop that optimizes meta-parameters, hyperparameters, architectures, or priors that shape the inner loop's behavior. In ML (Machine Learning) practice this appears in bilevel optimization, gradient-based meta-learning (e.g., MAML), hyperparameter tuning via differentiable or implicit gradients, hierarchical reinforcement learning where subpolicies learn within higher-level controllers, and modular continual-learning systems that separate fast plastic and slow stable components. Theoretically, nested learning raises questions about the stability and convergence of coupled optimizers, the propagation of gradients through truncated or implicit inner-loop dynamics, and the bias–variance tradeoffs introduced by multi-timescale updates. Practical techniques to make nested learning tractable include truncated backpropagation through inner updates, implicit function theorem or conjugate-gradient approximations for exact outer gradients, regularization of inner-loop updates to improve outer-loop signal, and careful scheduling of update frequencies; these choices affect sample efficiency, transferability, and robustness of the learned inductive biases.
First uses of nested, bilevel, or explicitly inner/outer loop formulations in ML trace back to early 2000s (concepts earlier in control theory and optimization), and the term and pattern gained broad popularity in the mid–late 2010s, accelerating after 2017 with the rise of gradient-based meta-learning (e.g., MAML) and scalable differentiable bilevel methods.