A chatbot is software that holds a conversation with people in text or voice, usually to answer questions or collect information. It may follow a scripted decision tree, match messages to predefined intents, or generate replies with a large language model. Most chatbots answer and route; they do not complete tasks in other systems.
"Chatbot" covers 60 years of technology, from ELIZA's keyword scripts in the 1960s to today's language-model assistants.1 The word says nothing about how the bot works, which is why two vendors can both sell a "chatbot" and mean very different products.
The useful questions are how it understands a message, where its answers come from, and whether it can act. A bot that can look up an order and issue a refund is no longer only a chatbot. It is an AI agent with a chat interface.
The same message, two kinds of chatbot
A customer writes to a telecom's chat. The first bot runs on a decision tree. The second uses a language model grounded in the company's help content.
Rule-based chatbot
Scripted- MatchFinds the keyword "plan" and starts the plan-change flow.
- MenuShows buttons: Upgrade, Downgrade, Compare plans, Something else.
- LoopThe customer clicks Something else and gets the same menu again.
Misrouted. The billing question is lost and the customer asks for a person or leaves.
LLM chatbot
Understands- UnderstandRecognizes two issues: a possible double charge and a question about the plan price.
- RetrieveFinds the help article on billing cycles and pro-rated charges after a plan change.
- ReplyExplains that a plan change mid-cycle can produce two partial charges, and offers to connect billing if the amounts do not match.
- Hand overPasses the conversation to a billing agent with the question summarized.
Understood and explained. Still no refund: checking the account and correcting it needs a person or an agent with billing tools.
Illustrative example. Neither bot can see the customer's bill, which is the line between a chatbot and an AI agent.
Three generations of chatbot
The U.S. Consumer Financial Protection Bureau separates rule-based chatbots, which use decision trees or keyword databases to trigger preset responses, from more complex ones that use machine learning or language models to generate replies.3 In practice there are three generations, and many deployed bots mix them.
Rule-based
Understands: keywords and button clicksA decision tree written by a flow designer. Predictable and cheap, but users must fit their problem into the menu. ELIZA, in 1966, already worked by matching keywords and applying transformation rules from a script.1
Intent-based (NLU)
Understands: a fixed list of intentsA classifier maps each message to one intent, such as "track order" or "reset password", and extracts details like an order number. Each intent has a scripted response.2 Accurate within its list, blind outside it.
LLM chatbotCommon today
Understands: open languageA large language model tuned to follow instructions generates the reply.5 It handles phrasing and multiple issues in one message, and needs grounding in company content to stay accurate.6
The step after the third generation is not a better chatbot but a different system: an agent that can call tools, check records, and complete the request. See agentic AI for that spectrum.
Where chatbots still fit
A chatbot is the right tool when the answer does not depend on the customer's own records, or when the next step is always a person. These are common uses across the industries below.
Pre-sale questions
Store hours, delivery options, size guides, and return windows. The answers are the same for everyone and change rarely.
question → retrieve → answerStructured intake
Collects the claim type, policy number, and photos before a handler sees the case, so the first human contact starts with complete information.
ask → validate → route to handlerOutage and status notices
Answers "is the network down in my area" from a status feed during an incident, absorbing the spike in identical questions.
detect topic → status feed → answerPolicy explanations
Explains baggage rules, change fees, and documents needed. Rebooking itself needs an agent with access to the reservation system.
question → policy → answer or hand overIllustrative examples. Where the customer wants something changed (a refund, a new date, a cancellation), an AI agent with tools resolves more of it.
Chatbot, AI agent, and related terms
| Term | What it names | Can act in other systems | Typical outcome |
|---|---|---|---|
| Chatbot | Any software that converses with people | Rarely, and only through fixed flows | An answer or a routed conversation |
| Conversational AI | The technologies behind natural dialogue (NLU, speech, generation) | Not by itself | A capability, not a product |
| Virtual agent | Contact-center term for a bot that handles a full contact | Sometimes | A contained contact |
| AI agent | A system that pursues a goal by calling tools | Yes, within permissions | A resolved request |
| IVR | A phone menu driven by keypad or speech | Only through scripted integrations | A routed or self-served call |
Where chatbots fail, and the controls
The CFPB's review of chatbots in consumer finance found customers caught in repetitive "doom loops" when their issue fell outside the bot's capabilities, left without a path to a human, and given inaccurate information by LLM-based bots.3 Each has a known control.
Doom loops
ControlDetect repeated intents or menus and hand over after two failed attempts. Measure how many conversations end in a loop, not only how many end without a person.3
No way to reach a person
ControlOffer a human route at any time and honor requests for one. Pass the transcript so the customer does not repeat themselves.
Invented policies
ControlAn LLM bot can state a rule that does not exist. In Moffatt v. Air Canada, a tribunal held the airline responsible for its chatbot's wrong refund advice, saying it makes no difference whether information comes from a static page or a chatbot.4 Ground answers in approved sources and test them against real questions.6
Undisclosed bot
ControlThe EU AI Act requires that people be informed when they interact with an AI system, unless it is obvious.7 State it in the first message.
Common questions
What is the difference between a chatbot and an AI agent?
A chatbot answers and routes. An AI agent can also act: it checks the order, applies the policy, and issues the refund through tools it is allowed to use. Many AI agents use a chat window, so the difference is in the back end, not the interface.
Is ChatGPT a chatbot?
Yes, in the broad sense: it converses in text and voice and generates replies with a large language model. With browsing, code execution, or connected apps it can also act, which moves it toward an agent.
Are rule-based chatbots obsolete?
Not entirely. For a short, fixed process with legal wording, such as consent capture or identity checks, a scripted flow is predictable and easy to audit. Many teams run scripted steps inside an LLM-based conversation.
Why do customers dislike chatbots?
Usually because of the failures above: the bot misunderstands, repeats the same menu, cannot do what was asked, or hides the way to a person. A bot that resolves the request or hands over quickly with context avoids most of the frustration.
References
- Weizenbaum, J. ELIZA: A Computer Program for the Study of Natural Language Communication between Man and Machine. Communications of the ACM 9, no. 1 (1966): 36-45.
- Google Cloud. Intents. Dialogflow ES documentation.
- Consumer Financial Protection Bureau. Chatbots in consumer finance. June 2023.
- Civil Resolution Tribunal of British Columbia. Moffatt v. Air Canada, 2024 BCCRT 149. Decision of 14 February 2024, via CanLII.
- Ouyang, L. et al. Training language models to follow instructions with human feedback. NeurIPS 2022.
- Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
- European Union. Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 50.