Prompt injection is an input validation problem wearing new language. An attacker’s goal is the same as any injection attack: get the system to treat untrusted data as an instruction. The fix borrows from the discipline that reduced SQL injection, applied to a boundary that no parser enforces.
OWASP’s Top 10 for LLM Applications ranks prompt injection first for exactly this reason. Unlike SQL, there is no perfect separator between instruction and data inside a natural-language prompt, so the mitigation is layered rather than absolute.
On this page
- Why there is no clean separator
- Direct vs indirect injection
- Layer the defenses
- Treat model output as untrusted too
- Test with real injection patterns
- Build the detection loop
- Where agents change the equation
- Common attack examples
- FAQ
Why there is no clean separator
A SQL query separates code and data with syntax a parser enforces. A language model reads system instructions, retrieved documents, and user text as one continuous stream of tokens. Nothing in the architecture guarantees the model will always respect which part was meant to be authoritative.
This is why prompt injection cannot be patched the way a single SQL injection bug can. It is a structural property of how current models process text, and defenses must assume some attempts will get through.
- No hard instruction/data boundary exists in the token stream.
- A model can be persuaded by well-crafted text, not just malformed input.
- Defenses must be layered, not singular.
Direct vs indirect injection
Direct injection happens when a user types an instruction meant to override the system prompt. Indirect injection is more dangerous: a malicious instruction hidden inside a document, webpage, or email that the model retrieves and processes as part of its normal task. The user never sees it and the filter rarely catches it.
Indirect injection matters most for agents with tool access, because the hidden instruction can direct the agent to exfiltrate data or take an unintended action, not just produce a bad answer the user can dismiss.
- Direct: the user is the attacker.
- Indirect: a third-party document is the attack vector.
- Indirect injection is harder to detect because the user did not write it.
Layer the defenses
No single control stops prompt injection. Combine input filtering for known attack patterns, clear separation of system instructions from retrieved content, least-privilege tool access so a successful injection has limited reach, and output validation before any action executes. Each layer covers a failure mode the previous one misses.
Privilege separation matters more than pattern matching. An attacker who fully controls the model’s text output should still be unable to cause damage if the surrounding system enforces what actions are actually possible.
- Filter and delimit at the input stage.
- Limit privileges at the execution stage.
- Validate at the output stage before any action.
Treat model output as untrusted too
A model that has been successfully injected will produce output that looks legitimate to a downstream system. Validate every model output the way a web application treats user input: check structure, allowlist expected actions, and require confirmation for high-consequence steps.
This is the single most effective mitigation for agents with tool access, because it does not depend on catching the injection attempt itself.
- Validate output structure before parsing.
- Require confirmation for irreversible actions.
- Log the full input and output for review.
Test with real injection patterns
Testing against a handful of obvious phrases catches almost nothing. Use a documented pattern library, rotate test cases as new techniques appear, and include indirect injection through documents the system would realistically retrieve.
Track a pass rate over time rather than treating one clean test run as proof of a fixed problem, since new bypass techniques appear regularly.
- Use a maintained pattern library, not ad hoc phrases.
- Include indirect injection via retrieved content.
- Retest after every model or prompt change.
Build the detection loop
Log prompts, retrieved content, and model output together so an analyst can reconstruct what happened after an incident. Alert on anomalous tool calls, unexpected data access patterns, or output that deviates from the expected task shape.
| Defense layer | What it stops | What it misses |
|---|---|---|
| Input filtering | Known attack phrases | Novel phrasing |
| Privilege limits | Damage from a successful injection | Data exposure within scope |
| Output validation | Unexpected actions executing | Subtle content manipulation |
Operating rule: Assume some injection attempts will reach the model. Design so a successful injection cannot cause a high-consequence action alone.
Where agents change the equation
A chatbot that can only produce text has a limited worst case: a bad answer reaches one user. An agent that can call APIs, write files, or send messages converts a successful injection into a real-world action. The same attack that was an annoyance becomes an operational incident.
This is why agent permissions and output validation belong in every injection defense plan. They cap the damage of the attempts that get through, which is the realistic assumption to design against.
- Text-only output limits the worst case to bad content.
- Tool access turns content risk into operational risk.
- Permission caps and output checks are the compensating controls.
Common attack examples
The practical patterns repeat. A resume that tells a screening assistant to recommend the candidate regardless of content. A web page that instructs a research agent to email its findings to an outside address. An email that turns a summarisation tool into a data-exfiltration channel. Each hides the instruction inside content the system was asked to process.
None of these look like attacks to a filter scanning for obvious phrases. That is why pattern-matching input filters rank lowest among the layers, and why privilege and output controls do the real work.
- Instructions hidden in documents the system retrieves.
- Content that redirects agent actions to attacker channels.
- Attacks that survive phrase-based filtering easily.
FAQ
Can prompt injection be fully solved?
Not with current model architectures. The goal is layered mitigation and limited blast radius, not a single fix.
Is indirect injection more common than direct injection?
Indirect injection is growing as more systems retrieve external content, and it is harder to detect since the attacker never interacts with the system directly.
Does limiting tool permissions really help?
Yes. It is the most reliable mitigation because it works even when the injection attempt succeeds at the text level.
Should every AI feature worry about this?
Any feature that processes untrusted text, including documents, emails, or web content, should assume injection attempts will occur.
How do teams test for prompt injection?
Maintain a pattern library, include indirect vectors, and retest after every model or prompt update.
Is a system prompt instruction enough protection?
No. It reduces easy attempts but is not a reliable control against a motivated attacker.
What should be logged for injection detection?
Full input, retrieved content, and model output together, so an analyst can reconstruct the sequence after an incident.
Related reading
- AI Security Needs Its Own Threat Model
- AI Agent Tool Permissions Need Least Privilege
- AI Output Handling Needs Downstream Validation
Does a chatbot without tool access need these controls?
A lighter set is enough: input filtering and output review. Privilege controls matter once the system can take actions.
How do we know if an injection succeeded?
Anomalies in tool calls, unexpected data access, or output that departs from the task shape are the usual signals. Prompt-level logging makes reconstruction possible.
What is the fastest mitigation to deploy?
Restrict tool permissions. It needs no model changes and caps the damage of any injection that gets through the input filters.
Should we blocklist attacker phrases?
Blocklists catch only known attempts and are trivially rephrased. Use them as one layer, never as the primary control.
Can prompt injection be detected before the model processes the input?
Sometimes, but never reliably. Treat detection as one signal among several and rely on privilege limits and output validation as the controls that always apply.
Do larger models resist injection better?
Newer models resist naive attempts better, but no model version eliminates the structural risk. Updated capabilities change the attack surface rather than removing it.
How does this relate to jailbreaking?
Jailbreaking aims to bypass content rules. Prompt injection aims to hijack the task. The techniques overlap and the same layered controls apply to both.
Conclusion
Prompt injection is unsolved at the model level, so the system around the model has to compensate. Filter input, separate instructions from data where possible, limit privileges, and validate output before anything acts on it.
Sources
More Stories
AI Access Logging Needs Prompt-Level Detail
Standard web access logs are not enough for AI systems. Prompt-level logging captures what an incident reconstruction actually needs.
Shadow AI Needs Discovery Before Policy
A policy written before discovery fails. Find actual AI use through network review, audits, procurement records, and non-punitive surveys.
AI Incident Response Needs a Model-Specific Playbook
A generic IT incident response plan misses AI failure modes. A model-specific playbook covers rollback, evidence, and decision rights.
AI Output Handling Needs Downstream Validation
Raw model output should never reach a database, shell, or user unvalidated. Apply the same discipline used for user input.
Training Data Poisoning Needs Source Control
Poisoning attacks corrupt training or fine-tuning data. Source control, provenance, and auditing reduce the risk before a training run.
AI Red Teaming Needs a Realistic Adversary
Red teaming an AI system differs from red teaming a network. Define the adversary first, then test, document, and feed findings back.