Guardrails

Guardrails sit on the edges of the model. On the input side they screen prompts for disallowed content, known jailbreak patterns or sensitive data before the model sees them. On the output side they inspect the response for policy violations, leaked secrets or unsafe instructions before it reaches the user or a downstream tool. Some are simple keyword or regular-expression filters; others are separate classifier models trained to spot problem content. Used well, they reduce the rate of obvious abuse and catch accidental leaks. They are a useful layer, in the same way a web application firewall is a useful layer in front of a web application.

The mistake is treating guardrails as the security control rather than one filter over a structural weakness. A pattern-matching filter is bypassed by rephrasing, encoding, or splitting a request across turns; a classifier has its own blind spots. More importantly, guardrails do nothing about the design problem underneath: if the model is wired to a tool that can move money and it is tricked by an indirect prompt injection, the output filter never sees the action, only the text. From the attacker’s side, a guardrail is an obstacle to route around, not a wall, and the systems that lean hardest on filters are usually the ones that skipped constraining what the model is allowed to do.

We test guardrails as filters and report their bypass rate honestly, but the more important finding is what the model can reach when a bypass succeeds: excessive agency (a model granted more tools and permissions than the task needs) turns a filter miss into real impact. We record which controls exist, how they were bypassed, and what the successful bypass could do. The recommendation is almost always to constrain capability first and filter second. This is part of how we test controls against real attacks.