· Hugo Berton
What is RAG (retrieval-augmented generation)?
RAG, retrieval-augmented generation, is a way to make a language model answer from your own documents rather than from memory. Before writing, the system searches your documents for the passages that matter, puts them in front of the model, and asks it to answer from them, ideally citing where each part comes from.
How it works
First, your documents are split into passages and indexed, usually as embeddings in a vector database so that a search finds passages by meaning, not just by keywords.
At question time, the system retrieves the closest passages, adds them to the prompt, and the model writes its answer from that material. Nothing is retrained: updating the knowledge means re-indexing the documents.
Why businesses use it
A model alone does not know your prices, your procedures or your contracts, and it may invent them. RAG grounds the answer in your sources and lets you check them.
Typical uses: an assistant that answers support questions from the product documentation, or one that finds the right clause in a contract template.
RAG or fine-tuning?
RAG adds knowledge at question time and keeps it in your documents. Fine-tuning retrains the model and is better suited to teaching a style or a format than to adding facts that change often.
Where it falls short
How documents are split matters: passages that are too long drown the answer, too short lose the context.
The index must be refreshed when the documents change, or answers go stale.
Without a set of test questions with known answers, there is no way to tell whether it really works. Build that set first.