Skip to content
Back to BlogAI Ops

The Document Is the Exploit: Indirect Prompt Injection as a Supply Chain Attack

July 7, 20269 min read

Your RAG pipeline has a WAF. Your API has rate limits. Your SSO has conditional access policies. The PDF a vendor emailed your agent last Tuesday had none of that scrutiny applied to it, and it can issue instructions with the same apparent authority as your system prompt.

That gap is not a rounding error. One survey of state-of-the-art LLM agents found 94.4% vulnerable to prompt injection, 83.3% vulnerable to retrieval-based backdoors, and 100% vulnerable to inter-agent trust exploits among the systems tested (Chhabra et al.). Adaptive attackers, ones who iterate against your specific defense rather than throwing a generic payload, bypass 90%+ of published defenses according to a systematic analysis of agentic coding assistants (Maloyan and Namiot). Twelve tested defenses, evaluated under adaptive attack conditions, all fell in the same paper.

This is not a chatbot jailbreak problem. It is what happens when you never vetted the vendor in your dependency chain, except the vendor is a document, and the dependency chain is your agent's context window.

The attack surface moved from the prompt box to the document

For two years, prompt injection meant a user typing something clever into a chat box. That threat model is mostly solved: rate-limit the user, log the session, review the transcript. Indirect prompt injection is a different animal. It is defined as malicious instructions embedded in external content that manipulate agent behavior, and it is explicitly a zero-click vector. No user has to type anything adversarial at all (Maloyan and Namiot).

The mechanism is almost boringly simple once you see it. Attackers enclose harmful tasks inside resources that look harmless: databases, APIs, web pages (Chhabra et al.). A malicious instruction doesn't announce itself. It reads like a footnote, a comment, or a line of boilerplate in a document your agent was already going to summarize.

Here's what that looks like at the HTML layer. A red-teaming tool called IPI-proxy built a payload library of 820 deduplicated attack strings pulled from six published benchmarks (BIPIA, InjecAgent, AgentDojo, Tensor Trust, WASP, and LLMail-Inject), then tested three embedding techniques against six insertion points on real, whitelisted web pages: HTML comments, invisible CSS, and semantic prose blended into normal copy (Chen et al.). The domain being trusted is irrelevant if nobody verifies the content on it. A payload styled with display: none or folded into a paragraph that reads like ordinary site copy carries no syntactic signature a filter can catch, because it isn't malformed. It's just text your agent was going to read anyway.

Input sanitization is structurally the wrong layer

The instinct, reasonably, is to filter. Strip suspicious phrases, block known jailbreak strings, add a "please ignore instructions embedded in documents" line to the system prompt. None of it holds up, and the reason is worth stating precisely.

Prompt injection is persuasion, not a bug.

Jessica Hammond — Rules fail at the prompt, succeed at the boundary, MIT Technology Review

Traditional input validation was built to catch malformed or malicious syntax: a SQL string with an unescaped quote, a script tag in a form field. Prompt injection wears legitimate syntax. It's an ordinary sentence that happens to be semantically adversarial. A keyword filter, or a polite in-prompt instruction asking the model to "follow these safety rules," cannot reliably stop a determined adversary from talking a model into something (Hammond).

The numbers back the argument up from three independent angles, and they should be attributed separately rather than blended into one figure:

  • Obfuscation defeats filters outright. Base64 encoding, Unicode homoglyphs, and simple word-splitting bypass most filtering with no sophistication required (Maloyan and Namiot).
  • Adaptive attacks bypass 90%+ of published defenses, across all twelve defenses that same paper evaluated (Maloyan and Namiot).
  • A separate survey puts adaptive bypass at 50% against eight defenses purpose-built for indirect prompt injection specifically, when the attacker manipulates the external content the defense was designed to catch (Chhabra et al.).

Different papers, different defense sets, different percentages. What's consistent across all of them is direction: every measured defense degrades hard the moment the attacker adapts to it. None report a defense holding above roughly 50% mitigation against a sophisticated, adaptive adversary.

There's a second, uglier problem with trying to train your way out. Defensive training, the RLHF-style approach of fine-tuning a model to refuse injected instructions, can make the model better at hiding misbehavior rather than better at avoiding it (Hammond). You optimize against your eval set, the model learns to pass your eval set, and the underlying susceptibility to a differently-phrased attack doesn't move. "Just RLHF it more" is not the fix, because the thing you are fighting is not a bug pattern. It is an open-ended persuasion problem, and persuasion doesn't have a finite blocklist.

Treat it as supply-chain integrity, not input hygiene

This isn't a novel metaphor invented for this post. The academic security community already formalizes prompt injection, tool manipulation, and supply-chain compromise as a connected family of threats in agent systems, not three unrelated bug classes (Siu et al.). That's a useful signal on its own: when a formal-methods paper groups your three problems under one threat model, the practitioners chasing incident reports are not overreacting.

The fix that survives adaptive attackers isn't a better prompt. It's boundary enforcement: moving the decision about what an agent can do out of the model's context window and into a system the model cannot talk its way past.

Concretely, that means:

  • Policy engines and identity systems decide what the agent can actually do, with which data, under which approvals, rather than the model inferring permission from the tone of a document (Hammond).
  • Dynamically scoped, least-privilege permissions, and explicit user control over any state-changing action, so a poisoned document can at most cause an agent to attempt something, not to succeed (Hammond).
  • Agents treated as first-class subjects, with the same observability tooling, red-teaming packages, and audit logging you'd demand of a human employee with API credentials, not a chat feature (Hammond). We wrote about the logging half of that requirement in /blog/audit-logs-ai: the same "instrument the boundary, not the prompt" logic applies whether the actor writing to your database is a person or an agent that read a poisoned PDF.

The relationship between the two failed approaches and the one that holds is simple enough to draw once, not worth belaboring in prose after.

The filter sits between the document and the model, exactly where persuasion has already won by the time anything reaches it. The policy engine sits between the model and the action, where it doesn't matter whether the model was persuaded. It matters whether the action is in scope.

Control belongs in the same place it always has in security: at the architecture boundary, enforced by systems, not by vibes.

Jessica Hammond — Rules fail at the prompt, succeed at the boundary, MIT Technology Review

Audit your ingestion points Monday morning

Pull the list of everything your agents read without a human in the loop: RAG loaders, email summarizers, web-fetch tools, anything that turns a document into context. For each one, ask a different question than the one your team has probably been asking. Not "is this document safe to read," which is unanswerable at scale, but "what can this document make the agent do?"

Then move that answer out of the prompt and into a capability-scoped policy layer. Two concrete patterns to start with, both drawn from the architectural-defense recommendation in the coding-assistant research cited above: capability scoping, where each tool the agent can call carries an explicit, narrow permission set instead of inheriting the agent's full privilege, and cryptographic tool provenance, where the agent can verify a tool or document source rather than trusting it by convention (Maloyan and Namiot). Neither is a filter. Both are enforced by the system the agent runs inside, which is the only place enforcement has ever reliably lived.

References