Prompt Injection Defense for AI Agents: The Controls That Actually Hold

The Short Answer

Prompt injection defense is an architecture problem, not a detection problem. The controls that hold are structural: never let a single agent session combine untrusted content, private data, and an outbound communication channel. Scope every credential to the one resource the session needs. Allow specific egress paths and methods instead of domains. Strip external image URLs before rendering. Classifiers buy you telemetry and slow an attacker down. They do not stop a motivated one.

Why the Layered Detection Stack Does Not Hold

The standard advice is to sanitize input, add a classifier, filter output, add a human approval step, and call it defense in depth. Every layer in that stack has now been measured.

In "The Attacker Moves Second" (Nasr, Carlini, Sitawarin and colleagues, October 2025), twelve published jailbreak and prompt injection defenses were attacked with gradient descent, reinforcement learning, random search, and human-guided exploration. Most were driven above 90% attack success, and most had originally reported near-zero. A parallel red-teaming competition with more than 500 participants and a 20,000 dollar prize pool defeated every challenge in the study.

One result from that paper is worth pinning to the wall. Spotlighting and Prompt Sandwiching, measured on AgentDojo with the benchmark's static attacks, showed attack success as low as 1%. Under the authors' search-based adaptive attack, both went above 95%.

The vendors agree. Microsoft's Azure AI Content Safety documentation states in its own troubleshooting notes that Prompt Shields may not catch all attack vectors and that you should always implement additional validation layers. OpenAI CISO Dane Stuckey wrote in October 2025, when ChatGPT Atlas shipped, that prompt injection remains a frontier, unsolved security problem, and that adversaries will spend significant time and resources finding ways to make the agent fall for it.

This is not an argument for skipping detection, only for knowing what you bought. Detection buys time, telemetry, and cost per attempt. Architecture buys a boundary.

The One Rule That Decides Your Agent Architecture

Two independent framings arrived at the same place in 2025, and the agreement is what makes them worth designing around.

Simon Willison's lethal trifecta names three capabilities that, combined, make an agent exfiltratable: access to private data, exposure to untrusted content, and the ability to communicate externally. Meta's Agents Rule of Two, published October 2025, says an agent must satisfy no more than two of three properties within a session: processing untrustworthy inputs, accessing sensitive systems or private data, and changing state or communicating externally. Meta is explicit that satisfying the rule is not a finish line, and that an agent needing all three requires human approval or another reliable validation.

Use this as the first gate in design review. If a session holds all three legs it is exploitable, and classifier tuning does not change that. Remove a leg before you ship.

The incident data agrees. Prompt injection holds first place in the OWASP Top 10 for LLM applications, 2026 edition, and Excessive Agency made the biggest move in that edition, from sixth to third. The list was built on incident evidence rather than expert voting alone: the OWASP GenAI Security Project, co-chaired by Steve Wilson, collected 7,714 incident records from public vulnerability databases and an AI harm database and classified the 6,639 that carried enough detail. The finger points at agency, not at detection quality.

What To Do When You Cannot Remove a Leg

Most useful agents need all three properties somewhere. Split the workflow, not the model.

The published catalogue of architectural defenses (Beurer-Kellner, Debenedetti, Tramèr and colleagues, June 2025) names six patterns: Action-Selector, Plan-Then-Execute, LLM Map-Reduce, Dual LLM, Code-Then-Execute, and Context-Minimization. Their shared principle: once an agent has ingested untrusted input, it must be constrained so that input cannot trigger any consequential action.

Three moves cover most of it:

1. Summarize untrusted content in a context with no credentials and no tools, then pass only structured fields to the privileged context. Raw retrieved text never reaches the planner.
2. Fix the tool sequence before untrusted data enters. If the plan can be rewritten by data the agent has not read yet, control flow already belongs to the attacker.
3. Compile the plan to code, and call unprivileged models only for text handling inside it.

Google DeepMind's CaMeL is the strongest published version of the third move. It extracts control and data flow from the trusted user query so untrusted data can never influence program flow, then adds capability-based access control on tool calls. It solves 77% of AgentDojo tasks with a provable security property against 84% for the undefended baseline. Seven points of utility for a design-level guarantee is the best trade published so far, and most teams refuse it without doing the arithmetic.

Your Egress Allowlist Is Probably Not Egress Control

Anthropic's May 2026 engineering post on containing Claude across products describes its own red team finding an egress path through an allowlisted domain. The injected instructions had Claude read other files in the workspace and upload them to Anthropic's own Files API using an attacker-supplied key. Across 25 retries it worked 24 times. The domain was approved. The data still left.

ForcedLeak, disclosed against Salesforce Agentforce in September 2025 at CVSS 9.4, is the worse variant. Noma Security chained an indirect injection through a Web-to-Lead description field that accepted 42,000 characters with a Content Security Policy allowlist that still trusted an expired domain. They bought that domain for five dollars and exfiltrated through an HTML image tag.

Build it the other way: allow specific paths and methods, not hosts. Log every outbound call with destination and payload size, alert on bodies above a threshold, and re-verify ownership of every allowlisted domain quarterly, because a domain that expires is a domain an attacker can buy.

Then handle the rendering surface, which is where the data walks out. Image-based exfiltration is the primitive in ForcedLeak and in EchoLeak (CVE-2025-32711, found by Aim Security in Microsoft 365 Copilot, where hidden instructions in a document made Copilot emit an image URL encoding sensitive data with no user click). Google's Gemini defense stack names markdown sanitization and suspicious URL redaction as one of its five layers. Strip external image URLs and rewrite links before render. This is frontend work, and it usually belongs to a team nobody invited to the AI security review.

Human-in-the-Loop Is the Weakest Control You Have

This control appears in every framework and every compliance narrative, and it is empirically the softest layer in the stack.

Anthropic's May 2026 telemetry showed users approving roughly 93% of permission prompts. At that rate an approval dialog is a log line, not a gate. The same post reports OS-level sandboxing producing an 84% reduction in prompts shown, and Claude Code auto mode catching roughly 83% of overeager behaviors before they execute. Both are structural controls doing the work the dialog was supposed to do. Simon Willison raised the same objection to CaMeL, which he otherwise called the first credible prompt injection mitigation he had seen that does not just throw more AI at the problem: ask a user to approve enough actions and they fall into a state where they say yes to everything.

The design rule: if a tool is high-consequence and low-frequency, such as payments, deletes, sending mail, or granting access, make it unreachable from any context that has seen untrusted input rather than gating it behind a confirmation. Reserve prompts for genuinely irreversible actions, and make those dialogs look different from routine ones.

How To Choose a Guardrail Without Fooling Yourself

Pick on false positive rate and latency, not on bypass rate. Every published bypass number was measured against static attacks and will not survive an adaptive one. False positives and latency are what your product lives with every day, and they decide whether the guardrail is still switched on in six months.

Here is what the published numbers actually buy, and where each comes from:

Rows two and three are the same control. That is the lesson: a published gain and an adaptive result are different measurements, and every row that has only been tested statically has a row three waiting for it.

Check the coverage boundary before you buy. Azure Prompt Shields is trained and tested on eight languages: Chinese, English, French, German, Spanish, Italian, Japanese, and Portuguese. An injection in a ninth, or in base64, or in image alt text, routes around it with no optimization at all.

Cost is not the constraint. Amazon Bedrock charges 0.08 dollars per 1,000 text units for its standalone prompt attack check. Teams skip the architecture because it is work, not because detection is expensive.

Demand an Attempt Budget With Every Number

Any attack success rate quoted without an attempt budget is marketing. The whole result in The Attacker Moves Second is the distance between one shot and many: defenses that published near-zero went above 90% once the attacker was allowed to adapt, and Spotlighting moved from 1% to above 95% on the same benchmark. Anthropic's red team needed 25 retries to land 24 exfiltrations, which reads very differently from a single-attempt figure.

So change the question. Not "what is the attack success rate", but "what is the attack success rate at 100 adaptive attempts, run by someone who has read the defense". Ask it of your vendor and of your own team, and write the budget next to the number in the report. If nobody can answer, what you have is a best case.

The corollary is that a model upgrade is not a security control. Frontier models are measurably harder to inject than small open ones, and that gap buys real time on the attacker's clock. It does not change what a session can reach once an injection lands.

Scope the Credential and Review the Config Files

Invariant Labs demonstrated in May 2025 that a malicious GitHub issue in a public repository could make an agent using the official GitHub MCP server leak private repository contents through a pull request it opened in the public repo. The MCP server code was not vulnerable. One token spanned both surfaces. Per-session single-repository scoping would have stopped it with no model change.

The Miasma worm campaign, reported in June 2026, disabled 73 Microsoft GitHub repositories across the Azure, Azure-Samples, Microsoft, and MicrosoftDocs organizations by planting agent config files that auto-execute when a developer opens the repo in an AI coding tool: SessionStart hooks in .claude/settings.json and .gemini/settings.json, a prompt injection in .cursor/rules/setup.mdc, and an auto-run task in .vscode/tasks.json. Treat those files, plus mcp.json, as executable code in review and block them in CI on untrusted branches.

The Checklist

Run this before an agent touches production data:

• No session holds untrusted input, private data, and outbound communication at once. If it must, the third leg sits behind a hard architectural boundary, not a dialog.
• Untrusted content is summarized in a credential-free, tool-free context, and only structured fields cross into the privileged one.
• Every tool credential is scoped to the session's single target repository, tenant, or mailbox.
• Egress is allowlisted by path and method, with payload size logged and thresholds alerted.
• External image URLs are stripped and links rewritten before any render.
• High-consequence tools are unreachable from injected contexts rather than confirmation-gated.
• Agent config files are code-reviewed and CI-blocked on untrusted branches.
• Per-tool-call audit logs exist, because injection does not throw. The agent completes the user's task and the attacker's, and every dashboard looks normal.
• You have run an adaptive red team at a fixed attempt budget and report attack success at that budget, not best case.

If you have one security week, spend it on blast radius. Scoped tokens, egress paths, and action allowlists beat a week of classifier tuning.

How We Approach This at SASID

Across 7 production AI systems in 5 industries, the ones that stayed out of trouble were the ones where the boundary was structural. A plain-English query engine for a cybersecurity platform has run 6+ months in production with zero bad queries. HIPAA-compliant retrieval over medical records cut insurance denial appeals from 30 to 60 minutes of specialist time to under 2 minutes. Call center QA reached 100% automated coverage where manual review had only ever reached under 5%. The delivery model is the same each time: proof of concept in days, production in 4 to 8 weeks, then 90 days of monitoring.

Get a Free Technical Assessment

If you are shipping an agent that touches customer data, we offer a free technical assessment: a 30-minute call about your architecture, then a written roadmap within 48 hours covering trust boundaries, credential scoping, egress control, and a red team plan. No obligation, and the roadmap is yours to keep. Book at sasid.ai.