How a RAG Pipeline Works End to End: Ingest, Retrieve, Generate

A RAG pipeline has two separate flows. The ingestion flow runs ahead of time and builds a searchable index from your documents. The query flow runs for every user question: it finds relevant chunks and asks the model to answer from them. Keeping these two flows separate in your mind makes the whole system easier to design and debug.

The ingestion flow

Ingestion turns raw files into searchable records. It usually has five steps.

1. Load. Read documents from files, a database, a wiki or an API. Extract plain text and keep useful fields such as title, URL and last-modified date.
2. Clean. Remove navigation bars, repeated footers and broken characters that add noise.
3. Chunk. Split each document into passages of a few hundred tokens. Chunks are the units you will retrieve.
4. Embed. Convert each chunk into a vector with an embedding model.
5. Store. Save the vector, the chunk text and the metadata in a vector index or database.

Ingestion is a batch job. It can be slow, and you can run it on a schedule or when documents change. The quality of everything downstream depends on it, because the model can only use what you indexed.

The query flow

The query flow must be fast because a person is waiting. It has these stages.

1. Understand the query. Optionally rewrite it, for example to remove chat context or fix spelling.
2. Embed the query with the same model you used for the chunks.
3. Retrieve candidates. Ask the index for the top 20 to 50 nearest chunks, optionally combined with keyword search and metadata filters.
4. Rerank. Use a more precise model to reorder the candidates and keep the best 3 to 8.
5. Build the prompt. Place the chosen chunks and the question in a prompt with clear instructions.
6. Generate. Call the language model and return the answer, ideally with citations.

A skeleton in code

The skeleton below names each stage as a function. Real implementations come later in the course, but seeing the shape helps you place each topic. Every function has one job, which makes each one easy to test on its own.

from dataclasses import dataclass

@dataclass
class Chunk:
    id: str
    text: str
    source: str

def load_documents(path: str) -> list[dict]:
    raise NotImplementedError  # read files, return [{"text": ..., "source": ...}]

def chunk_document(doc: dict) -> list[Chunk]:
    raise NotImplementedError  # split into passages

def embed(texts: list[str]) -> list[list[float]]:
    raise NotImplementedError  # call an embedding model

def store(chunks: list[Chunk], vectors: list[list[float]]) -> None:
    raise NotImplementedError  # write to the vector index

def retrieve(query: str, k: int = 20) -> list[Chunk]:
    raise NotImplementedError  # embed query, search index

def rerank(query: str, chunks: list[Chunk], top_n: int = 5) -> list[Chunk]:
    raise NotImplementedError  # reorder with a stronger model

def answer(query: str, chunks: list[Chunk]) -> str:
    raise NotImplementedError  # build prompt, call the LLM

In a minimal app you can skip rerank and filters. In a production app, each stage tends to grow: ingestion gets queues and retries, retrieval gets hybrid search, and generation gets citations and safety checks.

Where time and money go

• Ingestion cost is mostly embedding calls, paid once per chunk (and again if you change models).
• Query latency is usually dominated by the language model call. Embedding the query and searching the index take tens of milliseconds in a healthy setup.
• Query cost is mostly the tokens you place in the prompt, so retrieving fewer, better chunks saves money.

Where quality is won or lost

When an answer is wrong, find which stage failed before you change anything. Ask these questions in order:

1. Was the right information in the index at all? If not, the problem is loading, cleaning or chunking.
2. Was it in the top candidates? If not, the problem is the embedding model, the query, or missing keyword search.
3. Was it in the final prompt? If not, the problem is reranking or the number of chunks you keep.
4. Was it in the prompt but ignored? Then the problem is the prompt or the model.

This checklist stops you from tweaking the prompt when the real fault is a bad chunk boundary. We return to it in the debugging posts near the end of the course.

Keep both flows consistent

One rule catches many bugs: ingestion and querying must use the same embedding model and the same preprocessing. If you embed chunks with one model and queries with another, the vectors live in different spaces and the nearest neighbours are meaningless. Store the model name next to your index so you can check it.

Now that you can see the full pipeline, the next posts look at the building blocks one by one, starting with how RAG compares to other ways of giving a model new knowledge.