Prompt Injection › Summary

Prompt injection attacks exploit the inability of language models to reliably separate instructions from data when both arrive as text in the same context. The key takeaways from this chapter are:

• Direct injection targets the user input channel: an adversarial user crafts a message that overrides or supplements the operator's system prompt, either changing the model's behavior or extracting the system prompt itself. Multi-turn and compositional variants spread the attack across multiple turns to evade single-query detection.
• Indirect injection is the more dangerous variant: the attacker embeds malicious instructions in external content (web pages, emails, database records) that the model fetches as part of a task, without any direct interaction with the conversation. The confused deputy problem describes how this hijacks the model's elevated tool access on behalf of the attacker.
• RAG pipelines create a specific injection surface through corpus poisoning: an attacker who can write to the retrieval index can inject instructions that fire whenever a semantically relevant query arrives. Relevance-based targeting allows precise control over when the injection fires.
• Invisible injection techniques exploit the gap between how humans and models read text, using whitespace, Unicode steganography, HTML comments, and formatting tricks to embed instructions that visual review misses.
• Detection approaches include rule-based pattern matching, ML classifiers, and dual-model architectures. None are complete defenses, but they reduce the attack surface when combined. The precision-recall tradeoff is central: recall (catching injections) should be weighted more heavily than precision in most deployments.
• Architectural defenses are generally more reliable than detection: privilege separation, human-in-the-loop gates for high-stakes actions, read-only access for retrieval tasks, and ephemeral credentials limit blast radius even when injection succeeds.
• The problem is unsolved at a fundamental level. Privilege-level tokens, structured context formats, interpretability-guided interventions, and formal verification are active research directions, but no approach has demonstrated complete injection prevention while maintaining agent utility.