Infinite Curriculum

Infinite curriculum describes a training setup where a learning system generates its own unbounded stream of tasks instead of drawing from a fixed, human-curated dataset. Task difficulty scales automatically with the learner's current ability, usually through self-play or a setter-solver loop, so training never runs out of data and never plateaus at a fixed level of challenge.

The term appears in 2025 reinforcement learning work on large language models and agents. SPIRAL (Liu et al., arXiv:2506.24119) has a model play zero-sum games against improving copies of itself, producing what the authors call an infinite curriculum of progressively harder opponents. Verifier-Backed Hard Problem Generation (Lai et al., arXiv:2605.06660) extends this to mathematics by adding an independent verifier to a setter-solver pair, aiming for a nearly infinite curriculum of valid, difficult problems. ZeroGUI (Yang et al., arXiv:2505.23762) applies the same principle to GUI agents, using a vision-language model to propose new interface tasks from live screenshots.

The approach responds to a scaling limit: human-labeled problem sets are costly to produce and get used up once models surpass their difficulty. An infinite curriculum lets training continue past that ceiling, but it risks the task generator learning to game its own reward instead of producing genuinely harder problems, a failure known as reward hacking. An unrelated 2018 use of the same phrase appears in medical-image classification, where it meant ranking image patches by scale rather than generating self-play tasks.