Lethal Trifecta

Opening

The lethal trifecta is Simon Willison's term (2024) for the three capabilities that, when present together in an agentic AI system, create an unavoidable prompt-injection vulnerability: (1) access to untrusted input that contains instructions, (2) access to sensitive tools that can take actions, and (3) access to persistent state or memory that survives across invocations. Willison argued that any system with all three properties is fundamentally compromisable — an attacker who controls any of the inputs can steer the agent into misusing any of the tools. The term was re-anchored in 2026 by Boris Cherny at Y Combinator when describing why Anthropic built the prompt-injection classifier for Claude Code.

Mechanism

The vulnerability mechanism is straightforward. A model that reads untrusted instructions (web pages, email, file contents, Slack messages) cannot reliably distinguish "this is data to be acted upon" from "this is an instruction to be followed" — the prompt-injection attack exploits exactly this ambiguity. If that same model also has tool access (shell, file system, network calls), the attacker can craft inputs that instruct the model to call those tools against the user's interest. If the model also has persistent state, the attack survives across sessions. Willison's "lethal" framing is that all three together are not a risk to mitigate but a guarantee of compromise — you can layer defenses, but the model must act on every instruction it processes, including malicious ones. Cherny notes that by 2026, Claude Opus 5 is "not prompt injectable anymore" through a combination of model-side alignment, mechanistic-interpretability-based detection (Crystal's neuron-firing patterns), and runtime classifiers.

Tradeoffs

The trifecta's framing creates an uncomfortable design constraint: any agentic system that reads the web and has tool access is, by Willison's definition, lethal. The trade-off is between agent capability (read, act, remember) and security guarantees (none of those). Real-world systems often pick two of three — for example, an agent that reads the web and has tool access but no persistent state is compromisable in a single session but does not leak state across sessions; an agent with persistent state but no untrusted-input access is constrained but safer. The architectural trade-off is whether to design agents narrowly (two of three) or build layers of defense on top of all three (Anthropic's approach with Claude Code).

Open Questions

Whether models can be trained to reliably refuse instructions inside data they read (the "instruction hierarchy" research direction), which would break the trifecta's first leg. Whether mechanistic-interpretability classifiers (Crystal's neuron-firing approach) generalize across model families and attack patterns. Whether the trifecta framing applies symmetrically to non-agentic systems — is a search engine with no tool access but persistent state also lethal? Whether the trifecta becomes a regulatory concept (specific bans on systems exhibiting all three properties) or remains a design heuristic.