AI guardrails are the checks and limits placed around an AI system to keep its inputs, outputs, and actions within rules a company sets. They include screening what goes into the model, checking what comes out, restricting which tools and data it can use, enforcing policy limits, and requiring human approval above set thresholds.
Guardrails sit around the model, not inside it. Training methods such as RLHF shape how a model behaves by default; guardrails are separate code and configuration that a company controls, tests, and changes without retraining anything.6
For a chatbot, guardrails mostly control what it says. For an AI agent that can refund, cancel, or change records, they also control what it does, and that is where most of the design work goes.
Where guardrails sit in an agent
A guardrail is a checkpoint on the path from the customer's message to the agent's reply or action. Each checkpoint can pass, modify, block, or escalate.
Input checks
Screen messages and retrieved content before the model sees them: off-topic requests, abuse, personal data that should be masked, and attempts at prompt injection.1 Classifier models such as Llama Guard check prompts and responses against a risk taxonomy.7
Tool permissions
Give the agent only the functions and access it needs. OWASP traces "excessive agency" to excessive functionality, permissions, or autonomy, and recommends enforcing authorization in downstream systems rather than trusting the model to decide.3
Output checks
Validate replies and tool arguments before they reach a customer or another system: groundedness, tone, banned claims, format. OWASP lists unvalidated model output passed downstream as its own risk.2
Policy limits and approval
Hard rules the model cannot override, such as refund caps or no changes to account ownership, and a human approval step for high-impact actions.3
One risky request, with and without guardrails
A customer of an online retailer sends a message that mixes a real request with an attempt to manipulate the agent.
No guardrails
Complies- ModelTreats the instruction as legitimate context.
- Toolpayments.refund("48213", 900)→ refund issued
- ReplyConfirms a refund larger than the order value, plus a voucher.
Loss, and no record of why. The model was the only control, and text in a message overrode it.
With guardrails
Contained- InputFlags an instruction-override attempt. The message is kept, the instruction is treated as untrusted.
- Toolorders.get("48213")→ paid €84.90 · delivered
- LimitPolicy: refund capped at the order value. Automatic up to €150, above that a person approves.
- Actreturns.open("48213", reason="damaged")→ return label sent
- ReplyOffers a replacement or a refund of €84.90 once the photo of the damage is received.
Customer helped, policy kept. The attempt is logged for review.
Illustrative example. Amounts, thresholds, and tool names are placeholders a company would set for itself.
Guardrails are configuration you can read
Good guardrails are explicit and versioned, so compliance and support leads can review them without reading prompts. A simplified policy for a returns agent might look like this.
agent: returns
tools:
orders.get: { access: read }
returns.open: { access: write }
payments.refund:
access: write
max_amount: order_total
auto_approve_below: 150 # EUR; above this a person approves
accounts.update: { access: none }
input:
mask: [card_number, iban]
block: [instruction_override, off_topic]
output:
require: [grounded_in_sources]
forbid: [legal_advice, competitor_pricing]
on_trip: handover_with_contextIllustrative configuration, not a specific product's format. The values are placeholders.
Guardrails and neighboring controls
| Control | Where it acts | Who changes it | Example |
|---|---|---|---|
| AI guardrails | Around the model, at runtime | The company deploying the agent | Refund cap, input screening |
| Alignment and safety training | Inside the model, during training | The model provider | Refusing harmful requests by default |
| System prompt | In the model's instructions | The deploying company | "Only answer questions about orders." Helpful, but text can be overridden |
| Content moderation | On inputs and outputs | Either side | Blocking abusive language |
| Human in the loop | At decision points | Operations teams | A person approves refunds above a threshold |
| AI governance | Across the organization | Leadership, risk, compliance | Who may deploy which agent, and how it is audited |
A system prompt alone is not a guardrail for actions: the same text channel that carries it can carry an attack.
Guardrails by agent type
The right guardrails follow from what an agent can affect. Customer-facing agents that move money need the strictest ones.
Refunds and changes
Caps on amounts, approval above thresholds, and no write access to account ownership or payment details.
request → check limit → act or approveRegulated answers
Output checks block advice the company is not licensed to give and require answers about terms to cite the contract.
draft → policy check → send or rewriteAccurate offers
Prices and discounts come only from the pricing system. The agent cannot invent a discount code.
price lookup → validate → quoteRight documents, right people
Retrieval respects each employee's access rights, so an HR policy draft never reaches the wrong team.
query → filter by access → answerWhere guardrails fail
NIST's AI Risk Management Framework organizes this work into four functions: govern, map, measure, and manage.4 Guardrails belong to manage, and they only work if the other three are done.
Bypass by injection
ControlClassifiers miss new phrasings. Assume some attacks get through, and rely on tool permissions and limits that hold regardless of what the model was told.1
Over-blocking
ControlStrict filters refuse legitimate questions and push customers to human agents. Measure false refusals alongside missed violations.
Checks that drift from policy
ControlRefund rules and terms change. Version guardrail configuration with the policy it encodes and review both together.
Confident errors
ControlGenerative models can state false content with confidence, which NIST calls confabulation.5 Output checks for groundedness catch some of it; approval steps catch more.
No audit trail
ControlLog every blocked input, tripped check, and approval decision, so incidents can be traced and rules tuned.
Common questions
What are examples of AI guardrails?
Blocking off-topic or abusive inputs, masking card numbers, limiting an agent to read-only access, capping refund amounts, checking that answers are grounded in sources, and requiring a person to approve high-value actions.
Are guardrails the same as a system prompt?
No. A system prompt is an instruction to the model, and text in a message or document can override it. Guardrails are separate checks and permissions enforced in code, which hold even when the model is misled.
Do guardrails slow an AI agent down?
Each check adds some latency. Teams keep it low by running fast, inexpensive checks first, running some checks in parallel with the model, and reserving slower checks for high-risk actions.
Who should own AI guardrails?
The business owns the rules, such as refund limits and what the agent may say. Engineering implements and tests them. Compliance reviews them. All three should be able to read the configuration.
Can guardrails stop prompt injection?
Not completely. Input filters catch many attempts, but the dependable defense is limiting what the agent can do: narrow tool permissions, hard limits, and human approval for actions that matter.
References
- OWASP GenAI Security Project. LLM01:2025 Prompt Injection. OWASP Top 10 for LLM Applications, 2025.
- OWASP GenAI Security Project. LLM05:2025 Improper Output Handling. OWASP Top 10 for LLM Applications, 2025.
- OWASP GenAI Security Project. LLM06:2025 Excessive Agency. OWASP Top 10 for LLM Applications, 2025.
- NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. January 2023.
- NIST. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1). July 2024.
- Rebedea, T. et al. NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails. EMNLP 2023 (demo track).
- Inan, H. et al. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. Meta, arXiv, 2023.
- OpenAI. Guardrails, OpenAI Agents SDK. Official documentation.