The Anatomy of an Agent's Context: What the Model Actually Sees

An agent's context on any given call is made of a handful of parts: the system prompt, the tool definitions, the message history, the results of earlier tool calls, and any extra data you load such as memory or retrieved documents. Knowing these parts, and which of them grow during a run, is the first step to controlling them.

The parts in request order

With the Claude API, a request is rendered in a fixed order: tools first, then the system prompt, then the messages. Here is what each part does.

1. Tool definitions

Each tool has a name, a description and an input schema. These are sent on every call. A tool with a long description or a large schema may cost several hundred tokens each time. Twenty tools can add thousands of tokens before the user has said anything.

2. The system prompt

This holds the agent's role, rules and style. It is usually the most stable part and is a good candidate for caching (post 43).

3. Message history

Each user message and assistant reply is kept and re-sent. The API is stateless, so your code must send the whole history again each turn.

4. Tool calls and tool results

When the model calls a tool, the call is stored as an assistant content block and your result is stored in the next user message. These are part of the history. In coding or research agents, tool results are usually the largest and fastest-growing part.

5. Injected data

Anything your code adds: retrieved passages, user profile facts, memory notes, the current date, or a to-do list. You choose these, which makes them the most direct lever you have.

Which parts grow

Part	Size over a run	Who controls it
Tool definitions	Fixed per run	You
System prompt	Fixed (or slowly changing)	You
History	Grows each turn	You and the user
Tool results	Grows fast, can spike	Your tool code
Injected data	Varies per call	Your retrieval logic

Measure the parts in your own agent

The code below uses token counting to report how much each part contributes. It builds a small request and counts the pieces by sending them with count_tokens. This endpoint does not generate text, so it is a cheap way to inspect size.

import anthropic

client = anthropic.Anthropic()
MODEL = "claude-sonnet-5-5"

tools = [{
    "name": "get_weather",
    "description": "Get the current weather for a city.",
    "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]
system = "You are a travel assistant. Be concise and friendly."
messages = [{"role": "user", "content": "Is it warm in Lisbon today?"}]

def count(**kwargs):
    return client.messages.count_tokens(model=MODEL, **kwargs).input_tokens

only_messages = count(messages=messages)
with_system = count(system=system, messages=messages)
with_tools = count(system=system, tools=tools, messages=messages)

print("messages only:", only_messages)
print("system adds:", with_system - only_messages)
print("tools add:", with_tools - with_system)

The output shows three numbers. The first is the user message with any fixed overhead, the second is the cost of the system prompt, and the third is the cost of the tool definitions. Exact values depend on the model's tokenizer.

What this tells you

Once you know the sizes, you can reason about trade-offs. If tool definitions are the biggest fixed cost, you might load fewer tools per task. If tool results dominate, you should trim them. If history dominates, you need summarising or clearing. Most agents have one part that dwarfs the others, and fixing that part gives the largest gain.

Common mistake

People tune the system prompt for hours while 80 percent of their tokens are one oversized tool result repeated every turn. Always measure before optimising. The next posts show how to measure and how to fix each part.

Exercise

1. Log the token count of each part for ten real runs of your agent.
2. Find the largest part and the part that grows fastest.
3. Write down one change you could make to each.