An AI agent is a software system that uses a language model to reach a goal on someone's behalf: it reads the situation, chooses which tools to call, acts in other systems, and stops or hands over to a person when it reaches its limits. It is built from a model plus instructions, tools, memory, and guardrails.
The model is the smallest part of the decision. What makes an agent useful, or dangerous, is everything around it: what it is told, which systems it may read and change, what it remembers, and when it must stop and pass the case to a person. Two agents running on the same model can behave completely differently because of those choices.
This page covers the agent as a system you design, buy, and run. For the decide, act, and observe loop itself and how much autonomy to grant it, see agentic AI.
What an AI agent is made of
Textbook AI defines an agent as anything that perceives its environment and acts on it.2 A modern AI agent fits that definition in a specific way. A language model does the reasoning, and the rest of the system decides what the model sees, what it can do, and what happens when it gets something wrong. Anthropic calls the core building block an "augmented LLM": a model enhanced with retrieval, tools, and memory.1
Model
The large language model that reads the conversation and decides the next step. It is replaceable: many teams test two or three models on the same task.
Instructions
The role, the policies it must follow, the tone, and the cases it must never handle. In practice this is a long system prompt plus policy documents.
Tools
Functions the model may call: look up an order, search the knowledge base, change a booking, open a ticket. Each one is described to the model with a name and an input schema.7
Memory
What it keeps within the conversation (order number, what it already tried) and, where allowed, across sessions (preferences, open cases).
Guardrails
Checks outside the model: permission scopes per tool, amount limits, content filters, and validation of every action before it runs.
Handover
Rules for when to stop and pass the case to a person, with a summary so the customer does not repeat themselves. Anthropic describes human checkpoints and stopping conditions such as a maximum number of steps.1
What a tool looks like to the agent
The model never touches a database directly. It sees a list of tools, each with a plain-language description and a schema for its inputs, and it asks the surrounding software to run one. The definition below follows the format of Anthropic's tool use API.7
{
"name": "get_order_status",
"description": "Return the delivery status and tracking events for one order. Use when the customer asks where an order is. Read-only.",
"input_schema": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The order number, for example 48213"
}
},
"required": ["order_id"]
}
}Illustrative tool definition. The description is part of the agent's instructions: it tells the model when the tool applies and that it changes nothing.
The design work lives in these definitions. A read-only lookup can run freely. A refund tool needs an amount cap, an idempotency key, and a rule for when a person approves. Keeping read and write tools separate makes it easy to launch read-only first and add actions one at a time.
Types of AI agents in a company
The useful split is by who the agent serves. Customer-facing agents talk to the public and carry the brand, so they need the tightest guardrails. Employee-facing agents work behind the scenes, often with broader data access and a person reviewing the output.
Support agent
Handles order status, returns, payment questions, and booking changes in chat or email, and completes the change instead of linking a form.
identify → look up → act → confirmProactive agent
Starts the conversation when an event happens: a delayed parcel, a failed instalment, a canceled flight. It contacts the customer with a fix already prepared.
event → assess → offer fix → notifyShopping agent
Asks what the customer needs, searches the catalog and stock, compares options, and builds the basket. Common in electronics, beauty, and furniture retail.
clarify → search → compare → add to cartVoice agent
Answers phone calls in natural speech, often behind or instead of an IVR menu, for billing, claims, or rebooking in telecoms, insurance, and travel.
listen → understand → act → transfer if neededBack-office agent
Reads invoices, claims, or supplier documents, matches them to records, and posts clean cases while routing exceptions to a person.
extract → match → post or escalateKnowledge agent
Answers staff questions from policies, contracts, product data, and past tickets, with citations, and opens a request when someone must act.
search → cite → open requestIllustrative examples. Each type can be text or voice, and several can share tools through one orchestrating agent.
AI agent, chatbot, and copilot
Vendors use the three words loosely. The practical test is who decides the next step and who carries it out.
| System | Who decides the next step | Who acts | Typical use | Main limit |
|---|---|---|---|---|
| Rule-based chatbot | The flow designer, in advance | The customer or a support agent | FAQs, menus, form capture | Breaks on anything outside the script |
| LLM chatbot | The customer, turn by turn | The customer or a support agent | Answers from a knowledge base | Answers well but changes nothing |
| AI copilot | An employee, with suggestions | The employee | Drafting replies, summarizing tickets | Saves time per contact, not contacts |
| AI agent | The model, within permissions | The agent, through tools | Resolving requests end to end | Needs integrations, limits, and testing |
| RPA bot | A fixed script | The bot, through the user interface | Repetitive data entry | Cannot handle variation or language |
Where agents fail, and the controls
OWASP names "excessive agency" as a top risk for LLM applications: damaging actions taken in response to unexpected, ambiguous, or manipulated model output. Its root causes are excessive functionality, excessive permissions, and excessive autonomy.3 Each maps to a control you design up front.
Too many tools or permissions
ControlGive each tool the narrowest scope it needs. A tool that reads orders should not be able to cancel them. Write actions get their own tools and their own limits.3
Manipulated input
ControlAn email, review, or uploaded document can contain text that tries to redirect the agent.4 Treat retrieved content as data, never as instructions, and validate every action against policy before it runs.
Confident wrong answers
ControlGround answers in the company's own sources, require a source for policy statements, and route low-confidence cases to a person.
Bad handovers
ControlDefine the triggers (customer asks for a person, repeated failure, amount above a limit) and pass a summary so nobody starts over.
Undisclosed AI
ControlUnder Article 50 of the EU AI Act, people must be informed when they interact directly with an AI system, unless it is obvious.5 Say so at the start of the conversation.
How to judge whether an agent works
Measure outcomes, not conversations. An agent that talks to every customer but resolves few of them has moved work around without removing any.
- Resolution rate: the share of conversations closed without a person, checked by sampling that the customer's problem was actually solved.
- Containment versus resolution: a customer who gives up also counts as contained, so track repeat contacts within a few days.
- Quality: CSAT on agent-handled conversations compared with human-handled ones for the same request type.
- Cost per resolved conversation: model and infrastructure cost plus the human time spent on handovers.
- Action accuracy: refunds, changes, and cancellations audited against policy.
Common questions
Is ChatGPT an AI agent?
In a plain chat it is an assistant: it answers and you act. When it browses, runs code, or completes tasks in other applications for you, it works as an agent. The label depends on what the system is allowed to do, not on the product name.
What is the difference between an AI agent and a chatbot?
A chatbot answers questions. An AI agent also takes actions in other systems, such as issuing a refund or changing a booking, and decides which steps to take to reach the goal. Many AI agents use a chat window, so they can look the same from the outside.
Do AI agents replace customer service teams?
They take over high-volume, policy-driven requests and hand the rest to people with context attached. Teams shift toward complex cases, exceptions, and improving the agent's instructions and tools.
How long does it take to build an AI agent for customer service?
It depends on how many systems the agent must reach and how clean their APIs are. A common path is a read-only audit of the systems, a proof of concept on one request type with real conversations, then production with actions added one at a time.
References
- Anthropic. Building effective agents. December 2024.
- Russell, S. and Norvig, P. Artificial Intelligence: A Modern Approach, 4th edition. Pearson, 2020. Chapter 2.
- OWASP GenAI Security Project. LLM06:2025 Excessive Agency. OWASP Top 10 for LLM Applications, 2025.
- OWASP GenAI Security Project. LLM01:2025 Prompt Injection. OWASP Top 10 for LLM Applications, 2025.
- European Union. Regulation (EU) 2024/1689 (Artificial Intelligence Act), Articles 3(1) and 50.
- Model Context Protocol. Specification.
- Anthropic. Tool use with Claude. Claude API documentation.