What Is Context Engineering and Why It Matters for AI Agents

Context engineering is the practice of deciding exactly which information goes into a language model's context window on each call, and in what form. It covers the system prompt, the tool definitions, the conversation history, retrieved documents, tool results and any memory you load. The goal is simple: give the model the smallest set of information that still lets it do the job well.

Why the context decides the result

A language model has no memory of its own between API calls. Everything it knows about your task, your user and your rules arrives as text in the request. If a fact is missing, the model guesses. If the request is full of noise, the model may follow the wrong part of it. So when an agent behaves badly, the cause is very often the context, not the model.

This matters more for agents than for chatbots. A chatbot answers one question. An agent runs a loop: it calls a tool, reads the result, decides what to do next and repeats. Every turn of that loop adds more text to the context. After twenty tool calls the window can be full of old search results and file dumps that no longer help. Someone has to decide what stays, what is trimmed and what is saved for later. That decision is context engineering.

What context engineering covers

The topic is wider than writing a good prompt. In this course we will work through these areas:

• System prompts: how to write instructions that guide an agent without over-constraining it.
• Examples vs rules: when to explain and when to show.
• Long inputs: how to lay out big documents so the model can use them.
• Token budgets: planning how much of the window each part may use.
• Memory: short-term history, long-term stores and notes files.
• Compaction: summarising or clearing old content in long-running agents.
• Just-in-time retrieval: loading data only when the agent needs it.
• Tool output: returning results that are short and useful.
• Prompt caching: arranging the prompt so repeated parts are cheap.
• Coding-agent files: project instruction files such as CLAUDE.md.
• Measurement: proving that a context change actually helped.

A first example

Here is the same question asked with two different contexts. The task is to answer a customer's refund question. The code uses the official Anthropic Python SDK. Install it with pip install anthropic and set ANTHROPIC_API_KEY in your environment.

import anthropic

client = anthropic.Anthropic()

question = "Can I return the headphones I bought 40 days ago?"

def ask(system_text):
    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=2000,
        system=system_text,
        messages=[{"role": "user", "content": question}],
    )
    return "".join(b.text for b in response.content if b.type == "text")

print("--- no context ---")
print(ask("You are a helpful assistant."))

print("--- with policy context ---")
print(ask(
    "You are a support agent for an online shop. "
    "Policy: items can be returned within 30 days of delivery if unused. "
    "Headphones are covered by a 12-month warranty for defects. "
    "Answer in two sentences and name the policy you applied."
))

With no context, the model can only give generic advice, because it does not know your return window. With the policy in the system prompt, it can say the 30-day return window has passed and mention the warranty for defects. The model is identical in both runs. Only the context changed.

The guiding principle

A useful rule is to include the smallest set of high-signal information that makes the desired behaviour likely. More is not automatically better. Extra text costs money, adds latency and can distract the model. In the next posts you will see that long contexts can actually reduce accuracy, which is why trimming is as important as adding.

What to do next

1. Pick one agent or prompt you already have and print the full text of one request. Read it as if you were the model.
2. Mark every part as useful, unclear or noise.
3. Count the tokens of each part (post 5 shows how).
4. Keep this list; you will improve it as you work through the course.

Most teams discover that a large share of their context is leftovers: stale instructions, repeated tool output and long histories nobody trimmed. Cleaning that up is often the cheapest improvement available. For the basics of writing prompts themselves, see the LLM APIs track on this site; this course focuses on the system around the prompt.