What Actually Stops Prompt Injection: Defenses vs. Theater

In October 2025, a team spanning OpenAI, Anthropic, and Google DeepMind published a paper with a blunt title: “The Attacker Moves Second.” They took 12 recently published prompt-injection defenses, most of which originally reported near-zero attack success, and broke most of them at over 90% attack success under adaptive attack (arXiv:2510.09023, 2025). The people building the models told you, in their own names, that their own field’s defenses do not hold.

In the lethal trifecta I showed the gun: an agent that holds private data, reads untrusted content, and can talk to the outside is unconditionally exploitable. This post is the holster. The uncomfortable part is that most of the holsters on sale are cardboard.

Here’s the thesis. Detection-based defenses are theater. Every prompt-level filter (a classifier, a firewall, an instruction-hierarchy prompt, a fine-tuned refusal) is bypassable by an attacker who optimizes against it, and they all are. The durable wins are architectural: capability and data-flow control, design patterns that make injection structurally unable to reach a privileged action, and Meta’s Agents Rule of Two. Design as if detection will fail, because the strongest detector of 2026 does.

One boundary first, because it gets conflated constantly. Sandboxing bounds the blast radius: it is what the agent cannot do even when it is compromised. This post is about the other layer: data-flow control denies the path, so the untrusted input never reaches the privileged action in the first place. Isolation contains the explosion. Data-flow control keeps the trigger from connecting to the charge. You want both.

Key Takeaways

• A 2026 study by researchers at OpenAI, Anthropic, and Google DeepMind broke 12 published prompt-injection defenses, most originally near-zero attack success, at over 90% under adaptive attack; prompting-based defenses collapsed above 95%, training-based to 96-100% (arXiv:2510.09023, 2025).
• Sort every defense into two buckets. Detection asks “is this input malicious?” and loses to the attacker who moves second. Architecture asks “can this action reach the untrusted data?” and has a provable answer.
• CaMeL defeats injection by design, solving 77% of AgentDojo tasks with provable security versus 84% for an undefended system (arXiv:2503.18813, 2025).
• Meta’s Agents Rule of Two is the policy you can mandate Monday: within a session, grant no more than two of these three: untrusted input, sensitive access, external communication (ai.meta.com, 2025).

Why can’t you detect your way out of prompt injection?

You cannot, because the attacker moves second. A 2026 study by researchers at OpenAI, Anthropic, and Google DeepMind broke 12 published defenses, most originally reporting near-zero attack success, at over 90% attack success under adaptive attack (arXiv:2510.09023, 2025). Detection is a contest you enter already losing.

The asymmetry is the whole story. A defense ships as a fixed artifact. The attacker reads it, optimizes an input against it, and submits second. “Near-zero ASR” is a static-attack number: it measures how the filter does against payloads written before the filter existed. The paper’s title is its thesis. Whoever moves second wins, and in deployment the attacker always moves second.

The breakdown by defense family is worse than the headline. Prompting-based defenses like Spotlighting and Prompt Sandwiching showed adaptive attack success above 95%, and RPO-style methods landed at 96-98%. Training-based defenses fared no better: StruQ was broken “in every case we tested,” MetaSecAlign hit 96%, and Circuit Breakers reached 100% on HarmBench. Even filtering and detection systems collapsed: the adaptive attack fed the detector’s own confidence score back to the attacker and still cleared 90% (arXiv:2510.09023, 2025).

The mechanism, stated as a rule: a detector that sits in a loop the attacker can observe is a gradient the attacker can climb. This is not a quality problem that a better classifier solves. It is structural. As long as the defense is a fixed function and the attacker can probe it, the attacker finds the input that scores “safe” and carries the payload.

So the right question is not “which detector is best.” It is “what defense does not depend on detection at all?” A 2026 study from OpenAI, Anthropic, and Google DeepMind researchers broke 12 published prompt-injection defenses at over 90% attack success under adaptive attack, with prompting-based defenses collapsing above 95% and training-based defenses to 96-100% (arXiv:2510.09023, 2025). That is the empirical floor under everything that follows.

But PromptArmor hits sub-1% ASR. Isn’t that the counterexample?

It is the best counterexample, and it still does not change the verdict. PromptArmor, an off-the-shelf LLM prompting detector, achieves a false-positive rate, a false-negative rate, and an attack success rate all below 1% on the AgentDojo benchmark using GPT-4o, GPT-4.1, or o4-mini (arXiv:2507.15219, 2025). On paper it is excellent. On a static benchmark.

Let me steelman it properly, because strawmanning the strongest case is how you lose an argument you should win. PromptArmor is genuinely strong, cheap, and trivial to bolt on. If a fixed benchmark were the threat model, you would ship it and move on with your day. It is the most credible detection result of the year, and it deserves the credit.

Now the puncture. Every detector in the adaptive-attack paper also reported near-zero attack success before the adaptive attack. “Sub-1% on AgentDojo” is a static number against a fixed benchmark whose injections were not written to evade PromptArmor specifically. It is precisely the category the lab paper bypassed at over 90% once the attacker was allowed to optimize. To be exact about what the evidence does and does not say: PromptArmor was not among the 12 defenses that paper tested. The defensible claim is the one that matters anyway. PromptArmor belongs to the detection family, and its headline number is a static-benchmark number, which tells you nothing about an attacker who has your detector and a gradient.

Generalize the lesson and keep it. A defense’s benchmark number tells you how it does against the attacks in the benchmark. Read every “sub-1% ASR” claim as “sub-1% ASR against non-adaptive attacks,” because that is what it is. The strongest 2026 detector reports sub-1% false-positive, false-negative, and attack-success rates on AgentDojo, yet that is a static-benchmark result for a detection-family defense, the same category broken above 90% under adaptive attack (arXiv:2507.15219, 2025; arXiv:2510.09023, 2025).

Is detection therefore worthless? No. It raises cost and catches lazy attacks, the way a spam filter does. The error is treating it as a boundary instead of a speed bump. Defense in depth means detection on top of architecture, never instead of it. Hold that thought; we will give detection its honest job in a later section.

What does an architectural defense actually look like?

It looks like making the privileged action unreachable from the untrusted input, not inspecting the input. CaMeL (“Defeating Prompt Injections by Design”) extracts the control flow and data flow from the trusted user query, then uses capability-based access control to stop untrusted data from ever influencing a privileged operation. It solves 77% of AgentDojo tasks with provable security, versus 84% for an undefended system (arXiv:2503.18813, Google DeepMind, 2025).

The mechanism in plain terms: the trusted query defines a program, both its control flow and its data flow. Untrusted content (tool outputs, fetched pages) is tagged and carries capabilities that restrict what it can do downstream. When a tool that could exfiltrate is about to run, the policy checks whether the data feeding it is allowed to flow there. Injection inside the untrusted content cannot rewrite the control flow, because the control flow came from the trusted query, not from the data.

That is what makes it categorically different from detection. CaMeL never asks “is this input malicious?” It asks “is this data allowed to reach this action?” The second question has a provable answer. The first does not, which is why the entire detection family keeps losing.

The honest cost is real, so name it. The 77% versus 84% gap is a seven-point capability tax: some tasks legitimately need data flows the policy forbids without explicit user confirmation. CaMeL also requires you to structure the agent as an explicit program rather than a free-roaming loop. Provable security is not free. You pay for it in flexibility, and you should say so out loud when you propose it.

You do not need the full CaMeL system to get a slice of the benefit. The code-intelligence MCP I build answers code questions over a fixed query schema and has no outbound network capability at all. The tool that reads untrusted code is structurally incapable of exfiltration, because the capability is simply not present. That is hand-rolled Context-Minimization: the privileged channel does not exist, so there is nothing for injection to steer it toward.

// A capability-minimized tool: typed in, typed out, no egress leg.
// The model cannot be steered into exfiltration that the surface cannot perform.
type FindRefsInput = { symbol: string; repo: RepoId };
type FindRefsOutput = { refs: Array<{ file: string; line: number }> };

async function findReferences(input: FindRefsInput): Promise<FindRefsOutput> {
  // Reads a local, pre-built index. No fetch(), no socket, no shell.
  // Output is parsed-and-typed, not free text the model can be talked into.
  return index.lookup(input.symbol, input.repo);
}

That is the bridge from “academic provable security” to “what one engineer actually wired.” A narrow, typed tool is both a smaller attack surface and a smaller capability grant, which is the same argument that runs through why a tool’s schema is the prompt the agent reads. The shape of the tool is the capability you grant. Shape it so the dangerous action is not in scope.

The same split in table form, with what each bucket actually buys you:

Question the defense asks	Bucket	Named defenses	Holds under adaptive attack?
”Is this input malicious?”	Detection	Classifier, prompt firewall, instruction hierarchy, fine-tuned refusal	No. Bypassed at over 90% attack success (arXiv:2510.09023)
“Can this action reach the untrusted data?”	Architecture	Capability and data-flow control (CaMeL), the six design patterns, the dual-LLM split, the Agents Rule of Two	Yes. Structural, provable resistance (arXiv:2503.18813; arXiv:2506.08837)

The six design patterns that make injection structurally impotent

A 2026 paper from the CaMeL-adjacent group proposes six principled patterns for building agents with “provable resistance to prompt injection,” all resting on one idea: once an agent ingests untrusted content, constrain what it can do so injection cannot trigger a consequential action (arXiv:2506.08837, Beurer-Kellner, Buesser, Debenedetti, Fischer, Tramèr et al., 2025). None of the six asks whether the input is malicious. Each makes the dangerous action unreachable from the untrusted path.

Here are all six, with the one-line job of each:

1. Action-Selector Pattern. The agent can only pick from a fixed menu of pre-approved actions; it cannot author arbitrary tool calls. Injection cannot invent a new dangerous action, because the action space is closed.
2. Plan-Then-Execute Pattern. The agent commits to a plan before it sees untrusted content, and the plan cannot be rewritten by what it later reads. Injection arrives too late to change the control flow.
3. LLM Map-Reduce Pattern. Untrusted documents are processed in isolated sub-agents whose outputs are aggregated through a constrained interface. A poisoned document cannot reach across into a privileged channel.
4. Dual LLM Pattern. A privileged LLM never sees raw untrusted content; a quarantined LLM processes the untrusted content and returns only structured, non-actionable data. This is Willison’s original formulation, formalized. The untrusted content never gets a privileged channel.
5. Code-Then-Execute Pattern. The agent emits code, an auditable and analyzable artifact, which is then run under policy, rather than taking actions inline. The code can be statically checked before it executes.
6. Context-Minimization Pattern. Strip the context so untrusted content is removed or reduced before any privileged step. The less untrusted content reaches the privileged surface, the less injection can do.

The unifying principle is the whole point. Each pattern makes the dangerous action structurally unreachable from the untrusted path, which is the line between the right-hand bucket and the left. A 2026 paper formalizes six design patterns (Action-Selector, Plan-Then-Execute, LLM Map-Reduce, Dual LLM, Code-Then-Execute, and Context-Minimization) that give agents provable resistance to prompt injection by constraining what the agent can do after it ingests untrusted content (arXiv:2506.08837, 2025).

Each pattern trades generality for safety, and that is the honest limit. An Action-Selector agent cannot do open-ended work. A Plan-Then-Execute agent cannot adapt to what it reads mid-task. You buy security with capability, so choose the pattern that fits the task’s risk rather than reaching for the most flexible one by reflex.

The egress-proxy and sandbox pattern in Pylon is a coarse Dual-LLM-by-isolation. The agent that reads untrusted PR diffs has its privileged channel, outbound network, removed by the operating system rather than by a second model. It is cruder than the paper’s version and shippable today. Be honest about what it buys, though: the proxy bounds the damage, which is sandboxing’s job, but it does not prove the data flow safe the way CaMeL does. Isolation and data-flow control are different layers, and conflating them is how teams end up with one and believe they have both.

What is the Agents Rule of Two, applied to a coding agent?

Meta’s Agents Rule of Two gives the architectural posture a policy you can enforce without a PhD in capability systems: within a session, an agent should satisfy no more than two of three properties, which are processing untrustworthy input, accessing sensitive systems or private data, and changing state or communicating externally. If it needs all three, put a human in the loop (ai.meta.com, 2025).

Notice the mapping, because it is the bridge from the attack post to this one. The Rule of Two’s three properties are the lethal trifecta restated as a budget. The trifecta says all three together is fatal. The Rule of Two says pick at most two and design the third out of the session. Same geometry, expressed as a quota you can review.

Applied to a coding-agent setup, it gets concrete fast:

• A code-review agent reads untrusted PR content (property A) and has repo access (property B), so deny external communication (property C). No WebFetch, no outbound MCP, an egress proxy that denies all. That is Pylon.
• A research or fetch agent reads untrusted web pages (property A) and can communicate out (property C), so deny sensitive access (property B). Run it with no secrets and no private repos in context, so the worst it can leak is the public page it just read.

The honest caveat comes from Meta itself: the Rule of Two is “a supplement, and not a substitute” for least-privilege, and applying it “should not be viewed as a finish line for mitigating risk” (ai.meta.com, 2025). Critics note it gets harder to reason about across multi-agent and cross-tool flows, where capabilities recombine across a handoff and the trifecta reassembles in the seam between two individually-compliant agents. Treat it as the best practical first cut, not a proof.

Why does it beat a classifier as a mandate? Because it is a static, auditable property of how you wired the agent, not a runtime guess about whether an input is malicious. A reviewer can verify “this agent has no egress” in a config diff far more reliably than “this classifier catches injection.” Meta’s Agents Rule of Two says an agent should hold no more than two of untrustworthy input, sensitive access, and external communication per session, and calls itself a supplement and not a finish line, which makes it a wiring rule you can review rather than a detector you have to trust (ai.meta.com, 2025).

Where does detection still earn its place?

Detection is a speed bump, not a wall, and speed bumps have a job. After you have an architectural defense in place (capability control, a design pattern, the Rule of Two), a detector raises the cost of the lazy majority of attacks and buys you logs and alerts. The error is the inversion: shipping detection instead of architecture, then quoting the static-benchmark attack-success rate as if it were a security guarantee.

The correct layering reads as a stack, top to bottom by how much load it should carry:

1. Architecture first. Design the privileged action to be unreachable from the untrusted path: CaMeL, a design pattern, or the Rule of Two. This is the wall.
2. Isolation second. Bound the blast radius in case the wall is wrong: a sandbox, an egress proxy, no ambient credentials. This is the backstop.
3. Detection third. A classifier or firewall to catch the unsophisticated majority and to alert. Cheap, useful, never load-bearing.
4. Deterministic policy fourth. A hook that enforces the egress allowlist as code on every tool call, so the policy does not depend on the model’s cooperation. Prompt walls failed as governance; a hook runs as deterministic code.

Detection’s value is exactly a spam filter’s value. Nobody claims a spam filter makes email secure; it makes the inbox usable. A prompt-injection detector makes the agent’s day-to-day cleaner. It does not make the agent safe. The only danger is the marketing that conflates the two.

That is the precise meaning of “theater.” Theater is not “useless.” Theater is “presented as security when it is not.” A detector sold as defense-in-depth is honest. The same detector sold as “we solved prompt injection, sub-1% ASR” is theater. Build the stack so architecture carries the load, isolation bounds the damage, detection rides on top as a demoted speed bump, and a deterministic hook enforces the egress leg as code rather than as a request to the model.

A defense checklist you can ship this week

Five moves, ordered by leverage, that replace theater with architecture for a coding-agent and MCP setup. None of them require a vendor or a model upgrade. All of them are configuration you own.

1. Sort your current defenses into the two buckets. Anything that asks “is this malicious?” is detection, and it is theater if it is load-bearing. Anything that asks “can this action reach the untrusted data?” is architecture. Move budget from bucket one to bucket two.
2. Pick a design pattern per agent role. A review agent gets Action-Selector or Plan-Then-Execute, so the action space is closed and the plan is fixed before it reads anything. A fetch or summarize agent gets Dual-LLM or Context-Minimization, so untrusted content never touches a privileged channel (arXiv:2506.08837, 2025).
3. Apply the Rule of Two as a wiring rule, reviewed in config. No agent gets all three of untrusted input, sensitive access, and external communication in one session without a human in the loop. Make it a checklist item in the PR review of agent configs (ai.meta.com, 2025).
4. Make the egress leg deterministic, not promised. A PreToolUse hook that denies any tool call to a non-allowlisted host once the session has touched private data. Code, not a prompt (claude-code-hooks-substrate).
5. Keep the detector, demote it. Run the classifier as a logging speed bump on top of the architecture, never as the boundary. Stop quoting its benchmark attack-success rate as a guarantee.

The trifecta is the diagnosis. The dual-LLM split and the Rule of Two are the holster. The hook and the sandbox keep your finger off the trigger. Build as if detection will fail, because the strongest detector of 2026 did. Sort your defenses into detection and architecture, spend on architecture, isolate to bound the damage, and demote detection to a logged speed bump enforced by a deterministic hook.

One scope note before you ship this. Every move here defends runtime injection, where the malicious instruction rides in on untrusted content. It does nothing for a tool you installed on purpose that turns malicious. Poisoned tool descriptions are a supply-chain problem you fix at install time, with allowlists and version pinning, not with any runtime control on this list.

FAQ

Can prompt injection be detected reliably?

No. A 2026 study by researchers at OpenAI, Anthropic, and Google DeepMind broke 12 published defenses, most originally near-zero attack success, at over 90% under adaptive attack; prompting-based defenses collapsed above 95% and training-based defenses to 96-100% (arXiv:2510.09023, 2025). Even the strongest detector’s sub-1% is a static-benchmark number, not a guarantee against an adaptive attacker.

What is the best defense against prompt injection?

It is architectural, not detection-based: capability and data-flow control like CaMeL, or a design pattern that makes the privileged action unreachable from untrusted content, plus Meta’s Agents Rule of Two as the wiring quota. CaMeL solves 77% of AgentDojo tasks with provable security versus 84% undefended (arXiv:2503.18813, 2025). Architecture asks a question with a provable answer.

What is the dual LLM pattern?

A quarantined LLM processes untrusted content and returns only structured, non-actionable data, while a privileged LLM that can take actions never sees the raw untrusted content. It is one of six design patterns formalized in arXiv:2506.08837 (2025). The principle is that the untrusted content never gets a privileged channel, so injection has nothing to steer.

Isn’t a sub-1% ASR detector good enough?

Not as a boundary. Sub-1% on AgentDojo is a static-attack number that says nothing about an adaptive attacker who has read your detector (arXiv:2507.15219, 2025). Keep the detector as a speed bump on top of architecture, for logging and lazy attacks. Never ship it instead of architecture, and never quote its benchmark number as a security guarantee.

Detection is theater; architecture is the holster

The people building the models published the proof that detection does not hold (arXiv:2510.09023, 2025). Design as if detection will fail, because under adaptive attack it does, across prompting, training, and filtering alike.

The two buckets are the whole framework in one line. Detection asks “is this malicious?” and loses to the attacker who moves second. Architecture asks “can this action reach the untrusted data?” and has a provable answer. Sort every defense you own onto one side of that line, and move your budget to the right.

The stack is the takeaway: architecture first (CaMeL, a design pattern, or the Rule of Two), isolation to bound the damage, detection as a demoted speed bump, and a deterministic hook to enforce the egress leg. Spend on the right bucket. Keep the left one honest.

I build capability-minimized tools and run agents behind an egress proxy not because I trust a classifier, but because I assume the classifier will be bypassed, since the strongest one of 2026 was. The holster that holds is the one shaped so the gun cannot point at you. You have seen the gun; now build the holster out of architecture, not cardboard.