P(Doom)

P(doom) is an informal but increasingly prominent concept in AI safety discourse, referring to a researcher's subjective probability estimate that advanced artificial intelligence will lead to an existential or civilizational catastrophe — an outcome that either eliminates humanity or permanently and drastically curtails its long-term potential. Rather than a formally derived statistical measure, it functions as a shorthand for communicating one's overall risk assessment of transformative AI development, and individual estimates vary enormously across the research community, ranging from fractions of a percent to near-certainty.

The concept draws on expected value reasoning and decision theory: even a small P(doom) estimate, when multiplied against the magnitude of potential harm, can justify substantial investment in AI safety research. Researchers use it to anchor conversations about how seriously to treat tail risks — low-probability but catastrophic outcomes — when designing AI systems, governance frameworks, and alignment strategies. It implicitly encodes assumptions about timelines to powerful AI, the tractability of alignment, and the likelihood that misaligned systems could acquire sufficient capability to cause irreversible harm.

P(doom) estimates are inherently personal and epistemically uncertain, reflecting deep disagreements about the nature of intelligence, the difficulty of value alignment, and the robustness of human oversight mechanisms. Prominent AI safety researchers and lab leaders have publicly shared estimates ranging from below 5% to above 50%, and these figures often shift as new capabilities emerge or new alignment techniques are developed. Critics argue the concept can be misleading if treated as a precise probability rather than a rough intuition, while proponents contend that making such estimates explicit forces clearer thinking about assumptions and priorities.

The term gained significant traction in public and technical discourse around 2022–2023, coinciding with rapid advances in large language models and renewed mainstream attention to AI risk. It now serves as a cultural and rhetorical touchstone in debates about AI governance, the pace of capability development, and the urgency of safety research — making it a useful, if imprecise, lens for understanding how seriously different actors take long-term AI risk.