Retrieval-augmented generation (RAG)

RAG answers a question by fetching context before generating. The user’s query is turned into an embedding, matched against a vector store of pre-indexed documents, and the top results are inserted into the model’s prompt alongside the question. The model then answers using that retrieved text, which lets it cite current, private data it was never trained on and reduces confident invention. The trade is that the model now consumes content the developer did not write and may not have reviewed: whatever was indexed, and whatever a user is permitted to retrieve, becomes part of the model’s instructions for that turn.

Two security failures dominate. The first is that retrieved content is untrusted instruction: if an attacker can get text into the index, or into a document the pipeline pulls, they can attempt indirect prompt injection, and the model may act on it. The second is access control. Many pipelines index everything and enforce permissions weakly, so a user can retrieve, and have the model summarise, documents they should never see. From the attacker’s seat, the vector store is both an injection surface and a data-exposure surface, and the vector database behind it is frequently deployed with default, open access.

We test the retrieval pipeline as a whole: what can be indexed and by whom, whether retrieval respects the requesting user’s permissions, and what happens when a retrieved document carries instructions. Access control on the store and on the documents is assessed directly, because “the model only answers from our data” is not a control if any user can pull any document. We also check what the model is allowed to do with what it retrieves, because a pipeline that lets a retrieved document trigger a tool call turns a retrieval flaw into a real action. Filters are recorded as partial guardrails, not solutions. This is part of how we test a retrieval pipeline.