Sending Images and PDFs to an LLM API

Modern LLMs read images and PDFs as well as text. That makes it easy to build features like "extract the line items from this invoice" or "explain this error screenshot".

Image from a URL

const response = await client.messages.create({
  model: 'claude-opus-5-5',
  max_tokens: 16000,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'image', source: { type: 'url', url: 'https://example.com/dashboard.png' } },
        { type: 'text', text: 'Which metric dropped the most this week?' },
      ],
    },
  ],
});

Image from disk (base64)

import fs from 'node:fs';

const data = fs.readFileSync('error.png').toString('base64');

const response = await client.messages.create({
  model: 'claude-opus-5-5',
  max_tokens: 16000,
  messages: [
    {
      role: 'user',
      content: [
        { type: 'image', source: { type: 'base64', media_type: 'image/png', data } },
        { type: 'text', text: 'Explain this error and suggest a fix.' },
      ],
    },
  ],
});

PDF documents

PDFs are sent as a document block. The model sees both the text and the page images, so tables and charts are understood too.

import base64, anthropic

client = anthropic.Anthropic()
pdf = base64.standard_b64encode(open("invoice.pdf", "rb").read()).decode()

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=16000,
    messages=[{
        "role": "user",
        "content": [
            {"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": pdf}},
            {"type": "text", "text": "List every line item with quantity and price."},
        ],
    }],
)

Tips for better results

• Put the image or document before your question in the content array.
• Resize huge images; very large images cost more tokens without adding detail.
• Combine with structured outputs to get line items straight into typed objects.
• For a document you'll query many times, upload it once with the Files API and reference its ID.