Protecting RAG systems and AI agents against prompt injection

AI agents with a tool list, approvals and an audit log WZ-IT builds AI agents and internal assistants in which permitted actions are explicitly defined, critical steps require human approval and every action is logged. Incoming content from emails and documents is treated as data, not as instructions. Explore AI agents · Internal AI assistants

A RAG system reads documents; an AI agent reads emails, tickets and web pages. Any of these texts can contain instructions that the model follows even though no authorised person wrote them. That is prompt injection, and it again holds first place in the OWASP Top 10 for LLM Applications 2026. This article explains how direct and indirect injection work, why filters do not solve the problem and which architecture decisions limit the damage when an attack succeeds. As of October 2026.

Table of contents

• What prompt injection is
• Direct and indirect: where the instructions come from
• Why filters do not solve the problem
• How OWASP classifies it
• The critical combination: data, untrusted content, external effect
• Architectural controls at a glance
• Specifics of RAG systems
• Specifics of AI agents and MCP
• Detection tools as an additional layer
• Testing and monitoring

What prompt injection is

OWASP describes prompt injection as input that changes the behaviour of a language model in ways the application developer did not intend. The input can be user text, a retrieved document passage, a tool output, an image, an audio track or an entry in an agent's long-term memory (OWASP, LLM01:2026). It does not need to be readable or visible to humans.

The reason lies in how models work. System prompt, user question, retrieved documents and tool outputs all land in a single context window, as one stream of tokens. There is no structural boundary between "instruction" and "data". The UK NCSC puts it this way: under the hood of an LLM there is no distinction between data and instructions, there is only ever the next token (NCSC, Prompt injection is not SQL injection, December 2025).

This is the key difference from SQL injection. Parameterised queries solve SQL injection at the root. No equivalent currently exists for language models. OWASP, NCSC and NIST agree that no reliable mechanism to prevent prompt injection exists today. Defence is therefore architectural: the system is built so that a successful injection cannot cause serious harm.

Direct and indirect: where the instructions come from

The term indirect prompt injection goes back to a February 2023 paper by Greshake and others, which demonstrated such attacks against systems including the GPT-4-based Bing Chat (Greshake et al., arXiv 2302.12173). The pattern has stayed the same since: the attacker does not need to compromise the backend. They place text where the model will later read it, and the model does the rest with the user's permissions.

OWASP groups the sources of indirect injection by trust level (OWASP, LLM01:2026):

The third row is the one most often underestimated. An internal ticket system counts as trusted, yet it contains text entered by outsiders through a form. If an agent reads that ticket with write access to other systems, it acts on a stranger's instructions. On top of this come variants designed to evade filters: instructions in images, invisible Unicode characters, Base64-encoded text or languages a classifier was not trained on.

Why filters do not solve the problem

The obvious approach is a filter in front of the model that detects suspicious input. Such filters have their place, but they do not carry the security.

Adaptive attacks. Nasr and others tested 12 published defences against jailbreaks and prompt injection with attacks optimised specifically for each defence. For most of them the success rate exceeded 90 percent, although the defences had allowed almost no successful attacks in their original evaluations (Nasr et al., The Attacker Moves Second, arXiv 2510.09023).

The rate is not the measure. A filter that catches 95 percent of attacks is a failing grade in application security, because the attacker keeps varying the input until it lands in the remaining 5 percent (Simon Willison, The lethal trifecta).

Markers can be imitated. Techniques that label external content so the model recognises it as data demonstrably lower the success rate. For Microsoft's spotlighting technique, the authors reported a drop from more than 50 to below 2 percent (Hines et al., arXiv 2403.14720). OWASP points out, however, that such results come from non-adaptive tests and that an attacker who knows the marking scheme can mimic it.

The NCSC draws the conclusion: prompt injection will probably never be fully eliminated. The goal is to lower the likelihood and, above all, to limit the impact. The guiding principle: when an LLM processes information from a party, its privileges drop to those of that party (NCSC).

How OWASP classifies it

The current edition is the OWASP Top 10 for LLM Applications 2026, published in August 2026. The order has shifted noticeably compared with the 2025 edition. For prompt injection, these entries belong together:

OWASP describes the interplay explicitly: most high-impact incidents on record became severe because the injection landed in a system whose tools, permissions or rendering capabilities let the compromised model act on the user's behalf.

As soon as a model no longer just answers but calls tools and keeps memory across sessions, OWASP also refers to the OWASP Top 10 for Agentic Applications 2026 from December 2025. The most relevant entries there are ASI01 Agent Goal Hijack (the agent's goals are redirected), ASI02 Tool Misuse and Exploitation, ASI03 Identity and Privilege Abuse and ASI06 Memory & Context Poisoning.

The critical combination: data, untrusted content, external effect

Two design rules have become established as a pre-deployment check, and OWASP names both. Simon Willison describes the "lethal trifecta": an agent that simultaneously has access to private data, processes untrusted content and can communicate externally can be made to exfiltrate data (Willison, June 2025). In October 2025, Meta derived the Agents Rule of Two from it: within one session, an agent should have at most two of the three properties.

Legend: (A) untrustworthy inputs, (B) access to sensitive systems or private data, (C) changing state or communicating externally.

The practical value of the rule is that it forces a decision before anything is built. A task can often be cut so that one property disappears: the agent drafts the reply, a person sends it. Or the summary of external emails runs in a separate session without CRM access.

Architectural controls at a glance

No single control is sufficient. OWASP distinguishes between controls that lower the success rate and degrade against adaptive attackers, and controls that bound the impact of a successful injection. The second group carries the security.

Source of the structure: OWASP, LLM01:2026, Prevention and Mitigation Strategies.

The approach of fully separating control flow and data flow goes one step further. In CaMeL (Debenedetti and others, 2025), a privileged model plans the flow solely from the user's request; untrusted content is processed by a second model without tool access and can no longer change the flow. In the AgentDojo benchmark the system solved 77 percent of tasks with provable security, compared with 84 percent without protection (Debenedetti et al., arXiv 2503.18813). For individual agents with a clearly defined task, the same principle can be applied without a research framework: the flow is fixed in code, and the model fills in fields.

Specifics of RAG systems

A RAG system without tools cannot trigger actions, but it is not immune. Three paths remain open.

Manipulated answers. A poisoned passage in the index changes the answer for everyone whose question retrieves it. OWASP cites a study in which five poisoned documents in a knowledge base of millions of texts were enough for about 90 percent attack success. The countermeasure is control over who can bring content into the index: sources with external write access (ticket systems, shared folders, websites) are assessed and labelled separately.

Disclosure of context. Whatever is in the context can end up in the answer. That is why permissions belong before retrieval, not in the prompt. A passage the user may not see must not be retrieved in the first place. The mechanics are described in RAG with permissions.

Exfiltration through rendering. If the interface renders Markdown images, an injected instruction can make the model output an image whose URL carries conversation content as a parameter to an external server. The user sees only an image, or nothing at all. The only effective control here is an application setting: load no external images, or allow only targets from an allowlist.

Anyone systematically testing the answers of a RAG system should add poisoned documents to the test set. How such a test set is built is shown in RAG evaluation.

Specifics of AI agents and MCP

For agents, tool access determines the extent of the damage. OWASP describes cases in which an agent disclosed private repositories via a poisoned GitHub issue, or dumped a production database via an MCP server with administrative privileges, bypassing row-level security. In both cases the instruction sat in a channel an outsider could fill, and the agent executed it with the developer's permissions.

This leads to four rules for agents:

1. Enforce permissions in the target system. The agent calls regular interfaces with narrowly scoped rights. Whether an action is allowed is checked in the target system or in deterministic application code, not in the model. How identity, tool catalogue and approvals fit together is described in AI agents: permissions and approvals.
2. Approval with the exact action. Whoever approves sees the actual recipient, the actual text and the actual parameters, not a summary written by the model.
3. Memory as a privileged write operation. An entry in long-term memory affects every later session. Writes to it are logged and, if they contain instruction-like content, approved.
4. Treat MCP servers like software dependencies. Pin the version, verify the origin, review tool descriptions for hidden instructions. Tool descriptions land in the context and can themselves contain injections. More on this in MCP in the enterprise.

An MCP server that executes commands locally is effectively code execution on the host. That the gateways themselves can be attacked is shown by the LiteLLM vulnerability from September 2026.

Detection tools as an additional layer

Detection tools lower the likelihood of successful attacks and provide signals for monitoring. They do not replace any of the architectural controls.

As of October 2026. The 512-token context window means that longer documents have to be checked in sections. The Llama 4 Community License has to be considered in the licence review; what matters there is covered in Open LLM licences for commercial use.

Testing and monitoring

Test against adaptive attackers. OWASP recommends testing defences with testers who know the defence, and not accepting static success rates as proof. As a baseline, OWASP names the AgentDojo benchmark. For your own application, what matters most is a test set of poisoned emails, documents and tickets that match the sources actually connected.

Log. Every tool call is recorded with trigger, executing identity, parameters and result, together with the content that was in the context. Failed and rejected tool calls are a signal the NCSC explicitly names for monitoring. Langfuse records such runs as traces on your own infrastructure (What is Langfuse?).

Control changes. A model switch, a new system prompt or a new tool changes the attack surface. The injection test cases therefore belong in the same regression testing as the functional test questions.

What this means for your project

Prompt injection is not solved by choosing a model but by how the system is scoped. The questions to answer before building: Which sources can an outsider fill? Which data can the system reach? What effect can it have externally? Where a task needs all three, a human approval step belongs in the flow.

In WZ-IT AI agents, permitted actions are explicitly defined, critical steps require human approval, and every action is logged with a trace. Incoming content from emails and documents is treated as data; tool selection and approval logic never depend on the sender's text. For internal assistants, permissions take effect before retrieval. Models run on the AI Cube or on WZ-IT managed GPU servers. Support, consulting and implementation by WZ-IT.

How agents and automation fit in generally is explained in AI agents & automation. When agents query databases, Text-to-SQL is the next step, with its own requirements for read permissions and validation. A comparison of common frameworks from an operator's perspective is in the article on AI agent frameworks.

Sources

• OWASP Top 10 for LLM Applications 2026
• OWASP LLM01:2025 Prompt Injection
• OWASP Top 10 for Agentic Applications 2026
• NCSC: Prompt injection is not SQL injection (December 2025)
• Meta: Agents Rule of Two (October 2025)
• Simon Willison: The lethal trifecta for AI agents (June 2025)
• Greshake et al.: Indirect Prompt Injection (arXiv 2302.12173)
• Nasr et al.: The Attacker Moves Second (arXiv 2510.09023)
• Hines et al.: Spotlighting (arXiv 2403.14720)
• Debenedetti et al.: Defeating Prompt Injections by Design, CaMeL (arXiv 2503.18813)
• Debenedetti et al.: AgentDojo (arXiv 2406.13352)
• Meta: Llama Prompt Guard 2 model card
• Meta: LlamaFirewall (arXiv 2505.03574)
• NIST AI 100-2 E2025: Adversarial Machine Learning