AI Glossary Knowledge and retrieval

Retrieval-augmented generation (RAG)

RAGnoun · pronounced "rag"

  • Tidio Agents team
  • Updated November 19, 2026
  • 6 min read
  • 7 sources
Definition

Retrieval-augmented generation (RAG) is a method in which a language model first retrieves relevant passages from an external source, such as a knowledge base or a product catalog, and then writes its answer using those passages. The model answers from current, company-specific text it was given at query time, instead of relying only on what it memorized in training.

The term comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, which combined a trained text generator ("parametric memory") with a searchable index of documents ("non-parametric memory").1 The authors pointed to two problems with models that rely on training alone: they are hard to update, and they cannot show where an answer came from. Retrieval addresses both.

For a support or shopping agent, RAG is the difference between a model that knows what return policies usually say and one that quotes your return policy, as published this morning, with a link to the page.

How it works

The four steps of a RAG pipeline

A RAG system runs the same short pipeline for every question. Before any question arrives, the company's documents are split into passages (see chunking), converted into embeddings, and stored in an index, often a vector database.

QueryRetrieveAugmentGenerateanswer + sourceshelp center · policies · catalog · order data
The RAG pipeline. Retrieval decides what the model can see, so most quality problems start there.

Query

The customer's message, often rewritten into a clean search query that includes context from earlier turns ("it" becomes "the blue sofa, order 48213").

Retrieve

The system finds the passages most similar in meaning to the query. Dense retrievers that match embeddings beat keyword search (BM25) by 9 to 19 points on top-20 passage accuracy in open-domain question answering.2 Many production systems combine both.

Augment

The best passages are inserted into the prompt with an instruction such as "answer only from the sources below, and say so if they don't contain the answer."

Generate

The model writes the reply from those passages and can cite them. Lewis et al. found that RAG models produced more specific and factual language than a model without retrieval.1

Example

One question, with and without retrieval

A customer asks an online retailer's assistant about returns. The model is the same in both lanes. Only the second can see the retailer's documents.

Model alone

Memory
CustomerI bought a sofa in your Black Friday sale. Can I still return it? It arrived 20 days ago.
  1. RecallDraws on general patterns from training data: many retailers allow 14 or 30 days.
  2. Reply"Most furniture can be returned within 30 days, so you should be fine. Just pack it in the original box."

Plausible and wrong. This retailer extends returns for sale items but excludes made-to-order sofas. The model had no way to know.

RAG

Retrieved
CustomerI bought a sofa in your Black Friday sale. Can I still return it? It arrived 20 days ago.
  1. QueryRewrites to: return policy, furniture, sale purchase, 20 days after delivery.
  2. Retrievekb.search(query, top_k=4)→ returns-policy §3 · holiday-sales-terms · made-to-order-items
  3. AugmentAdds the three passages to the prompt with the rule: answer only from these sources.
  4. ReplyExplains the extended return window for sale items, asks whether the sofa was made to order (those can't be returned), and links both policy pages.

Correct, current, and checkable. The answer cites the pages it used, so a person can verify it.

Illustrative example. Policy names and terms are placeholders, not any real retailer's rules.

Compared

RAG, fine-tuning, and long prompts

Teams usually weigh RAG against fine-tuning and against pasting everything into a large context window. They solve different problems and are often combined.

ApproachWhat changesBest forUpdating factsCan cite sources
RAGWhat the model sees at query timeFacts that change: policies, prices, stock, order dataEdit the source document; the next answer uses itYes, the retrieved passages
Fine-tuningThe model's weightsTone, format, domain language, a narrow task done the same way every timeRetrain the modelNo, knowledge is blended into the weights
Long promptThe prompt, with every document pasted inSmall, stable document setsEdit the promptPartly, but accuracy drops for text buried mid-prompt4
Search onlyNothing in the modelUsers who want documents, not answersRe-indexReturns the documents themselves

A common production setup fine-tunes for tone and task, and uses RAG for facts.

In practice

Where RAG does the work

RAG fits wherever the right answer lives in a document or record the company controls and updates.

Customer support

Policy answers that match the policy

A telecom's agent answers roaming, billing, and contract questions from the current tariff documents, not from what tariffs looked like when the model was trained.

question → retrieve tariff → answer with link
AI search

Search that answers

A marketplace's help search returns a direct answer built from several help articles, with the articles listed underneath.

query → retrieve top passages → summarize → cite
Shopping

Advice from the real catalog

An electronics retailer's shopping agent retrieves product specs and stock before recommending, so it never suggests an item that doesn't exist.

needs → retrieve products → compare → recommend
Internal knowledge

One place to ask

An insurer's claims staff ask about procedures and get answers drawn from internal manuals, with the clause cited.

ask → retrieve manual → cite clause
Risks

Where RAG goes wrong

RAG reduces hallucination but does not remove it. Research on dialogue systems found that retrieval substantially reduced knowledge hallucination,5 yet a model can still misread a passage or answer from memory when retrieval finds nothing useful.

Wrong passage retrieved

ControlMeasure retrieval on its own, with a test set of real questions and the passages that should answer them. Fix chunking and ranking before touching the prompt.

Outdated or conflicting sources

ControlGive every document an owner and a review date. Remove old versions from the index instead of leaving both.

Answer goes beyond the sources

ControlInstruct the model to answer only from the passages and to say when they don't cover the question. Score faithfulness on live conversations.6

Data leakage

ControlApply the user's access rights at retrieval time, so an agent never retrieves another customer's records or internal-only files.7

Poisoned content

ControlAccept documents only from trusted sources and audit the index for hidden instructions. Treat retrieved text as untrusted input that cannot change what the agent is allowed to do.7

FAQ

Common questions

Is RAG better than fine-tuning?

They do different jobs. RAG gives a model facts it can cite and that you can update by editing a document. Fine-tuning changes how the model behaves, such as tone or output format. Most customer-facing agents use RAG for facts and, where needed, fine-tuning for style.

Does RAG stop hallucinations?

It reduces them, because the model answers from retrieved text instead of memory. It does not stop them. The model can still misread a passage or fill a gap when retrieval misses, so teams measure faithfulness and set the agent to say "I don't know" or hand over.

Do I need a vector database for RAG?

Not always. Small document sets can use keyword search or a simple embedding index. A vector database helps when you have many documents, need fast similarity search, or want to filter results by metadata such as market, language, or customer.

What is the difference between RAG and grounding?

Grounding is the goal: tying an answer to verifiable sources. RAG is one common way to achieve it, by retrieving those sources and giving them to the model before it answers.

Sources

References

  1. Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
  2. Karpukhin, V. et al. Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020.
  3. Gao, Y. et al. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv, 2023.
  4. Liu, N. F. et al. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the ACL (accepted 2023).
  5. Shuster, K. et al. Retrieval Augmentation Reduces Hallucination in Conversation. arXiv, 2021.
  6. Es, S. et al. Ragas: Automated Evaluation of Retrieval Augmented Generation. arXiv, 2023.
  7. OWASP GenAI Security Project. LLM08:2025 Vector and Embedding Weaknesses. OWASP Top 10 for LLM Applications, 2025.

Tidio Agents

Build an agent that resolves, not only answers

We design and build custom AI agents for customer service, shopping and back-office work, on your infrastructure, with the code handed over to you. Projects start with a 21-day private AI audit.

Request a call