How to Count Tokens with the Claude API Before You Send a Request

To count tokens with the Claude API, call client.messages.count_tokens() with the same model, system, tools and messages you plan to send. It returns an input_tokens number without generating any text. This lets you check a prompt against a budget before you pay for a real call.

Why not guess or use another tokenizer

Token counts depend on the tokenizer of the specific model. A library built for another provider will give numbers that are close but not exact, and the gap can be large for code or non-English text. Since Claude models can differ from each other too, count with the model you will actually call.

Basic usage

import anthropic

client = anthropic.Anthropic()

result = client.messages.count_tokens(
    model="claude-sonnet-5-5",
    system="You are a precise technical writer.",
    messages=[
        {"role": "user", "content": "Explain what a context window is."}
    ],
)
print(result.input_tokens)

The result is the number of input tokens the request would use. It includes the system prompt and the message, plus any fixed overhead the API adds.

Counting documents and tools

You can include tool definitions in the count, which is helpful because they are easy to forget. Here is a helper that counts a text file as it would appear in a prompt.

import sys
import anthropic

client = anthropic.Anthropic()
MODEL = "claude-sonnet-5-5"

def count_file(path: str) -> int:
    with open(path, encoding="utf-8") as f:
        text = f.read()
    result = client.messages.count_tokens(
        model=MODEL,
        messages=[{"role": "user", "content": text}],
    )
    return result.input_tokens

for path in sys.argv[1:]:
    print(f"{count_file(path):>8}  {path}")

Run it as python count.py README.md docs/guide.md. You get one line per file with its token count. This is a quick way to find which documents are too big to include whole.

Compare estimates with real usage

After a real call, the response's usage object reports the true numbers. Comparing the two is a good habit when you start, because it builds trust in your estimates.

import anthropic

client = anthropic.Anthropic()
MODEL = "claude-sonnet-5-5"
messages = [{"role": "user", "content": "Name three uses of a hash map."}]

estimate = client.messages.count_tokens(model=MODEL, messages=messages).input_tokens

response = client.messages.create(model=MODEL, max_tokens=2000, messages=messages)

print("estimated input:", estimate)
print("actual input:   ", response.usage.input_tokens)
print("output:         ", response.usage.output_tokens)

The two input numbers should match or be very close. The output count is only known after the call, so budget for it using your max_tokens value as an upper bound.

Using counts as a guard

A simple guard stops you from sending an oversized request. Set a limit in your code and take action when it is exceeded, such as trimming history.

MAX_INPUT = 60_000

def guarded_count(client, model, **kwargs):
    n = client.messages.count_tokens(model=model, **kwargs).input_tokens
    if n > MAX_INPUT:
        raise ValueError(f"Prompt is {n} tokens, over the {MAX_INPUT} budget")
    return n

Things to remember

• Token counting is a separate request. It is fast and does not generate text, but it is still a network call, so count in development tools and sampling, not on every production request unless you need to.
• Counts can change when you change models. Re-baseline after a model upgrade.
• Include tools and system in the count; leaving them out under-reports.
• For large images or documents, count them too, since they are converted to tokens as well.

Where token counting fits in a workflow

Counting is most useful at three moments. The first is design time: before you commit to putting a document or a set of examples in your prompt, count it and see what share of your budget it takes. The second is test time: run your counting helper over a sample of real conversations and look at the distribution, not just the average. An agent that averages 20,000 tokens but occasionally reaches 150,000 needs a different plan from one that is steady. The third is runtime: use cheap checks, such as the usage numbers already returned by every call, to trigger trimming or summarising when history crosses a threshold.

You do not need to call the counting endpoint on every turn of a production agent. Each response already reports the input tokens it consumed, and you can keep a running figure from that. Use the counting endpoint when you need a number before sending, for example to decide whether a candidate prompt fits.

Questions people ask

Does the count include the model's reply?

No. count_tokens reports input tokens only. The reply is limited by max_tokens and counted in output_tokens afterwards.

Why does a tiny prompt show more tokens than I expected?

The request format adds a small amount of fixed overhead for roles and structure, and tool definitions add more. This is normal. What matters is that your estimate and the real usage agree, which they should.

Can I count a whole conversation?

Yes. Pass the full messages list, including earlier assistant turns and tool results, exactly as you would in a normal call. This is the best way to see how big a long-running session has become.

Next step

Once you can measure, set up budgets for each part of the context. Post 14 shows how to plan them.