RAG: Chat With Your Documents

RAG stands for Retrieval Augmented Generation. That is a big name for a simple idea.

First fetch the right pages. Then let the model answer using only those pages.

It is an open book exam. The model does not have to remember everything. It just reads the page in front of it.

This is how you make a model answer questions about your own files, and stop it from making things up.

The whole pipeline in one picture

You already know every piece. Today we just plug them together.

Step 1: Load, split, embed, store

The file becomes chunks. The chunks become vectors. The retriever is ready to fetch.

Step 2: A prompt with two holes

The prompt needs a hole for {context} and a hole for {question}.

The line about saying I do not know is the anti-hallucination rule. It tells the model to stay inside the book.

Step 3: Glue the pages into one string

The retriever returns a list of Documents. The prompt wants one string. A tiny helper fixes that.

It joins each chunk with a blank line between them.

Step 4: The RAG chain

The dictionary at the start runs two things at once.

The question goes into retriever | format_docs and comes out as context. The same question passes straight through as question.

Both land in the prompt, then the model, then the parser gives you a clean string.

Step 5: Show your sources

A good answer says where it came from. Keep the documents and return them next to the answer.

assign adds an answer key but keeps the context list. The metadata tells you which file each chunk came from.

Tuning chunk size

If answers miss details, your chunks may be too small. Try 800 or 1000.

If answers wander or cost too much, your chunks may be too big. Try 300.

There is no perfect number. Test with five real questions and pick what works.

Why this stops hallucinations

Without RAG, the model answers from memory. Memory can be wrong or out of date.

With RAG, the model reads your actual text and is told to stay inside it.

It can still make small mistakes. But it is far less likely to invent a whole fact.

Remember: RAG is fetch first, then answer. The retriever brings the pages, the prompt says use only these, and the model writes the reply.