AI Glossary Models and how they work

Large language model (LLM)

LLMnoun · pronounced "el-el-em"

  • Tidio Agents team
  • Updated November 19, 2026
  • 7 min read
  • 9 sources
Definition

A large language model (LLM) is a neural network trained on very large amounts of text to predict the next token in a sequence, which lets it read, write, summarize, translate, and reason over language. It is the engine inside modern chat assistants and AI agents, but on its own it only produces text: it does not look anything up or take actions.

The key distinction for a buyer is between the model and the system built around it. An LLM is a general text engine. A support chatbot or an AI agent is a product that wraps an LLM with instructions, company data, tools, permissions, and checks. Most of what decides whether a customer gets a correct answer sits in that wrapper, not in the choice of model.

How it works

How a large language model is built

Almost every current LLM is a transformer, an architecture that uses a mechanism called attention to weigh how each part of the input relates to every other part.1 The model works on tokens, small pieces of text, and it is built in three broad stages.

1. Pretraining

The model reads a large share of the public web, books, and code, and learns to predict the next token. Doing that well forces it to absorb grammar, facts, styles, and patterns of reasoning. GPT-3, trained this way, had 175 billion parameters and could do new tasks from a few examples in the prompt.2

2. Instruction tuning

A pretrained model continues text; it does not follow requests. Developers fine-tune it on examples of instructions paired with good answers, so it learns to act as an assistant.

3. Preference tuning

People rank alternative answers, and the model is tuned toward the preferred ones. In OpenAI's InstructGPT work, outputs from a 1.3 billion parameter model tuned this way were preferred to those of the 175 billion parameter GPT-3.3

4. Use (inference)

In production the model receives a prompt and generates a reply token by token. Each new token is fed back in as input until the answer is complete.

PromptTokensTransformerNext tokenappend and repeatreply completeinstructions · conversation · retrieved documents
Generation is a loop. The model never sees the whole answer in advance; it commits to one token at a time.
Example

The same model, with and without a system around it

A customer asks an online fashion retailer about a return. Both lanes use the same LLM. Only the second one can see the order and the policy.

LLM alone

Predicts
CustomerI bought a jacket three weeks ago and it doesn't fit. Can I still return it?
  1. GenerateWrites the most likely answer for a typical retailer: "Most stores accept returns within 30 days."
  2. ReplySounds confident. The model has no access to this retailer's policy, which is 14 days for outerwear.

Fluent, plausible, wrong. The customer is promised a return the company will refuse.

LLM inside an agent

Grounded
CustomerI bought a jacket three weeks ago and it doesn't fit. Can I still return it?
  1. Toolorders.find(customer_id)→ jacket · delivered 22 days ago
  2. RetrieveFinds the returns policy: outerwear within 14 days, store credit up to 30 days.
  3. GenerateThe LLM writes the reply from these facts, not from memory.
  4. ReplyExplains that a refund is no longer possible, offers store credit, and starts the return if the customer agrees.

Correct and actionable. The answer cites the real policy and the real order.

Illustrative example. The policy terms and tool names are placeholders, not any retailer's real rules.

Capabilities

What an LLM can and cannot do in a business

LLMs are strong at language and weak at facts they were never given. That splits business tasks into two groups.

  • Reliable: understanding messy customer messages in many languages, classifying intent, extracting fields from documents, summarizing long threads, drafting replies in a set tone, and turning a request into a structured tool call.
  • Reliable with grounding: answering policy and product questions, when the system retrieves the right documents and tells the model to answer only from them. This is retrieval-augmented generation.4
  • Unreliable on its own: order status, prices, stock, account balances, or anything that changed after training. The model has no live data unless a tool supplies it.
  • Not its job: exact arithmetic over many records, enforcing business rules, and deciding what it is allowed to do. Code and permissions should do that, with the model calling them.
Compared

LLM, chatbot, and AI agent

The three are often used as synonyms. They sit at different layers of the same stack.

TermWhat it isKnows your data?Can act?In a support team
Large language modelA trained model that generates textOnly what is in the promptNoThe engine behind the other two
ChatbotAn application that holds a conversationIf connected to a knowledge baseRarely, and only along fixed pathsAnswers FAQs and hands off
AI agentA system that uses an LLM to pursue a goal with toolsYes, through retrieval and APIsYes, within permissionsChecks the order and issues the refund
Small language modelA compact model, often distilled from a larger oneOnly what is in the promptNoFast intent routing or extraction at volume
In practice

Where LLMs do the work inside an agent

In each case below the LLM handles the language, and the surrounding system supplies data, rules, and actions.

Customer support

Understand and resolve

Reads a complaint about a delayed parcel or a double charge, identifies the intent, and drafts the resolution from order and payment data.

understand → fetch data → resolve → reply
Shopping

Turn needs into a search

Converts "a quiet dishwasher that fits under my counter" into filters, compares the results, and explains the trade-offs.

interpret → search catalog → compare → advise
AI search

Answer from your content

Answers questions on a telecom's plans or an insurer's policies from approved documents, with the source shown.

retrieve → answer with citation
Back office

Read documents

Extracts amounts, dates, and parties from claims, invoices, and contracts in several languages into structured fields.

read → extract → validate → post
Internal knowledge

Summarize for employees

Summarizes a long customer history before a person takes over, so the customer does not repeat themselves.

collect → summarize → hand over
Lead generation

Qualify in conversation

Asks qualifying questions in natural language, by chat or voice, and writes the answers into the CRM.

ask → structure → update CRM
Risks

LLM risks and the usual controls

Hallucination

ControlGround answers in retrieved documents and live data, instruct the model to say when it does not know, and test on real transcripts before launch.5

Prompt injection

ControlText inside an email, review, or web page can carry instructions the model may follow.6 Treat retrieved content as untrusted and give each tool the narrowest permission.

Data exposure

ControlDecide what customer data reaches the model and where it runs. Mask personal data the task does not need, and keep logs under your own retention rules.

Inconsistent output

ControlThe same prompt can produce different answers. Use structured outputs for anything a system consumes, and score answers continuously, not once.

Transparency duties

ControlUnder the EU AI Act, people must be told when they are interacting with an AI system unless it is obvious.9

FAQ

Common questions

Is ChatGPT a large language model?

ChatGPT is an application. It runs on OpenAI's GPT family of large language models and adds a chat interface, memory, and tools such as web search on top.

What is the difference between an LLM and generative AI?

Generative AI is the whole category of models that create content, including images, audio, and video. LLMs are the part of it that works with text and code.

Do LLMs learn from my company's conversations?

Not by default. A deployed model's weights stay fixed. It only uses your data if you put it in the prompt, connect it through retrieval, or deliberately fine-tune a model on it.

Which LLM is best for customer service?

There is no single answer. Test two or three models on your own transcripts and languages, and compare resolution rate, latency, and cost per conversation. The system around the model usually matters more than the model.

Can an LLM run on our own infrastructure?

Yes. Open-weight models can run in your own cloud or data center, which keeps data inside your environment. The trade-off is that you operate and scale the hardware yourself.

Sources

References

  1. Vaswani, A. et al. Attention Is All You Need. NeurIPS 2017.
  2. Brown, T. et al. Language Models are Few-Shot Learners. NeurIPS 2020.
  3. Ouyang, L. et al. Training language models to follow instructions with human feedback. NeurIPS 2022.
  4. Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
  5. Kalai, A. T. et al. Why Language Models Hallucinate. 2025.
  6. OWASP GenAI Security Project. LLM01: Prompt Injection. OWASP Top 10 for LLM Applications, 2025.
  7. Bommasani, R. et al. On the Opportunities and Risks of Foundation Models. Stanford CRFM, 2021.
  8. Anthropic. Context windows. Claude documentation.
  9. European Commission. AI Act, Article 50: Transparency obligations. Regulation (EU) 2024/1689.

Tidio Agents

Build an agent that resolves, not only answers

We design and build custom AI agents for customer service, shopping and back-office work, on your infrastructure, with the code handed over to you. Projects start with a 21-day private AI audit.

Request a call