Overhang

Overhang refers to the disparity between the computational resources actually expended during model training and the theoretical minimum required to achieve a given level of performance. When a model is trained with significantly more compute than necessary, it often exceeds baseline performance expectations — a phenomenon that becomes especially relevant when algorithmic improvements or hardware advances suddenly make it possible to extract far more capability from existing compute budgets. The concept is closely tied to the idea that the frontier of AI capability is shaped not just by raw compute, but by how efficiently that compute is used.

The mechanics of overhang become clearest when viewed through the lens of scaling laws. Research has shown that model performance scales predictably with compute, data, and parameters. When training runs use compute inefficiently — for example, by under-training large models or using suboptimal architectures — there exists latent performance that could be unlocked simply by redistributing the same resources more effectively. Conversely, when a new algorithmic breakthrough dramatically reduces the compute needed for a given capability level, previously trained models may be found to have significant overhang: they were overtrained relative to what was strictly necessary, yet that excess may have conferred unexpected robustness or generalization.

Overhang has taken on particular significance in discussions about AI safety and forecasting. If large amounts of compute have already been spent training models that are less efficient than they could be, a sudden algorithmic improvement could rapidly close the gap between current and frontier capabilities without requiring additional hardware investment. This creates a kind of stored potential — a reservoir of latent capability that could be released quickly, making capability jumps harder to anticipate and govern.

Practically, understanding overhang helps researchers and organizations make better decisions about training runs, resource allocation, and model deployment. It also informs policy discussions about compute governance, since the relationship between compute expenditure and capability is not always linear or predictable. As the field matures and efficiency research accelerates, overhang will remain a key lens for interpreting the gap between what AI systems currently do and what they could do with better optimization.