Jailbreak (LLM)
In AI security, an LLM jailbreak is an input crafted to make a model ignore its own safety policy and produce output it was aligned to refuse. It is constantly confused with prompt injection, but the two are different: a jailbreak defeats the model’s policy, while injection hijacks the surrounding application’s instructions.
How it works
A jailbreak targets the alignment of the model itself. Techniques vary: role-play framings that reposition the request as fiction, instructions to answer in a format the safety training did not cover, gradual escalation across a conversation, or encodings that slip past a filter looking for keywords. The common thread is that the attacker is arguing with the model’s refusal behaviour, not with the application. That is the line that separates it from prompt injection: injection makes the model follow the attacker’s instructions in place of the developer’s, whereas a jailbreak makes the model break the constraint the model provider trained into it. A system can suffer both, and the fixes are different.
What goes wrong
Treating the two as one problem leads to the wrong defence. Teams add an input filter to stop “jailbreaks” and then discover it does nothing against an indirect prompt injection through their retrieval pipeline, or they harden the application prompt and leave the model happily role-playing its way past its own policy. From the offensive side, jailbreaks are also unstable: a phrasing that works today is patched by the provider, so a defence measured against a fixed list of known jailbreaks gives false comfort. The real question is not whether one clever prompt works once, but what the model is allowed to do when it does comply, which is a design question.
Where this shows up in an audit
We assess the model and the application as two layers. For the model we probe whether policy can be bypassed and, more usefully, what harmful output actually reaches a user or a downstream system when it is. We record whether the system prompt or other sensitive context can be extracted (system prompt leakage), because a leaked prompt makes further bypasses easier. Findings are framed by impact and by which layer failed. This is how we assess model and application safety together rather than testing prompts in isolation.