Contextual Retrieval: Better Chunks for Better RAG

A chunk like "Revenue grew 3% over the previous quarter" is useless on its own: which company, which quarter? Contextual retrieval, a technique published by Anthropic, fixes this by adding a short explanation of where the chunk sits in its document before embedding it.

The idea

Original chunk:
  "Revenue grew 3% over the previous quarter."

Contextualised chunk:
  "From ACME Corp's Q2 2025 earnings report, discussing results versus Q1 2025.
   Revenue grew 3% over the previous quarter."

Both the embedding and the keyword (BM25) index are built from the contextualised text. In Anthropic's experiments, this reduced failed retrievals substantially, and combining it with reranking reduced them further.

Generating the context with an LLM

import anthropic
client = anthropic.Anthropic()

PROMPT = """<document>
{doc}
</document>
Here is a chunk from the document:
<chunk>
{chunk}
</chunk>
Write 1-2 sentences situating this chunk within the overall document,
to improve search retrieval. Answer with only the context."""

def contextualise(doc, chunk):
    response = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=300,
        messages=[{"role": "user", "content": PROMPT.format(doc=doc, chunk=chunk)}],
    )
    context = next(b.text for b in response.content if b.type == "text")
    return f"{context}\n\n{chunk}"

Keeping the cost down

• The whole document is repeated for every chunk, so put it in a cached prefix: each later chunk reads it from cache at a fraction of the cost.
• Use a small, fast model for this step; it's a simple summarisation task.
• Run ingestion in batches; it doesn't need real-time responses.

Things to remember

• Show the original chunk, not the added context, when you cite sources to users.
• Re-run contextualisation when a document changes, since the context may change too.
• Measure recall@k before and after on your own evaluation set.