In AI and rationalist circles, "Moloch" symbolizes destructive coordination failures—scenarios where competing entities, driven by self-interest or competition, make choices that collectively lead to negative or suboptimal outcomes, even when all participants are aware of the harm. This concept draws heavily on game theory, illustrating how misaligned incentives between groups or individuals can lead to catastrophic results. In AI safety and alignment discussions, Moloch represents the potential dangers of poorly aligned artificial intelligence systems that could exploit weaknesses in human coordination, leading to outcomes that harm humanity, such as an arms race in AI development. It's used to highlight the importance of addressing global coordination issues to prevent existential risks.
The metaphor of "Moloch" was popularized in modern times by the 2014 essay Meditations on Moloch by Scott Alexander (from Slate Star Codex). Alexander used the ancient god Moloch, historically associated with child sacrifice, as a symbol for systems that destroy value through competition and misaligned incentives. Since then, the term has gained traction in AI safety, existential risk, and rationalist discourse.
Scott Alexander is the primary figure who popularized the Moloch metaphor in modern intellectual circles. His essay helped tie ancient mythological references to contemporary challenges in AI development, game theory, and coordination problems. Other thinkers in the rationalist and effective altruism communities, like Eliezer Yudkowsky, have expanded on the concept within the context of AI safety and existential risk.