Multi-Turn Conversations: Keeping the History Yourself

An LLM API does not remember anything between calls. To hold a conversation you keep the list of messages in your own code (or database), append each new user turn and each assistant reply, and send the full list every time. This is the single most important pattern in chat applications.

The append-and-resend loop

import anthropic

client = anthropic.Anthropic()
history = []

def chat(user_text: str) -> str:
    history.append({"role": "user", "content": user_text})
    reply = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1000,
        system="You are a friendly coding tutor. Keep answers short.",
        messages=history,
    )
    text = "".join(b.text for b in reply.content if b.type == "text")
    history.append({"role": "assistant", "content": text})
    return text

print(chat("My name is Priya and I'm learning Go."))
print(chat("What did I say my name was?"))
print(chat("Suggest a first project."))

The second answer will say Priya because the first exchange is included in the second request. Remove the two history.append lines and the model forgets her name immediately.

Why cost grows each turn

Each request carries the entire history as input tokens. If every turn adds about 300 tokens, turn 10 sends about 3,000 input tokens, and the total input across the conversation grows roughly with the square of the number of turns. Output tokens stay small, but input tokens add up. Prompt caching and history trimming, covered later, are the main tools to control this.

Where to store history

• In memory: fine for scripts and demos, lost on restart, and a problem once you run more than one server.
• A database row per message: the usual choice. Store conversation ID, role, content, timestamp and token usage.
• A fast cache such as Redis: good for short-lived sessions. Persist to a database if you need an audit trail.

Store what you sent, not just what you displayed

If a reply contains non-text blocks (thinking, tool calls), the API expects those blocks back on the next turn in some cases. Saving only the visible text will break those flows later. For plain chat, text is enough. Decide early, and store the full content list if you might use tools.

Handling a failed turn

If the call fails, do not leave the user message dangling in history without a reply, or the next request will end up with two user turns in a row. Append the user message only after a successful reply, or remove it on failure.

def safe_chat(user_text: str) -> str:
    candidate = history + [{"role": "user", "content": user_text}]
    reply = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=800,
        messages=candidate,
    )
    text = "".join(b.text for b in reply.content if b.type == "text")
    history.append({"role": "user", "content": user_text})
    history.append({"role": "assistant", "content": text})
    return text

Storing a conversation in a database

A simple schema is one table of conversations and one of messages. The messages table holds a conversation ID, a sequence number, the role, the content, and the token usage for assistant turns. When a new request comes in, load the messages ordered by sequence, call the API, and insert both new rows in a single transaction. Using a sequence number rather than relying on timestamps avoids ordering bugs when two requests arrive close together.

import sqlite3
import anthropic

db = sqlite3.connect("chat.db")
db.execute("""CREATE TABLE IF NOT EXISTS messages (
    conversation_id TEXT, seq INTEGER, role TEXT, content TEXT,
    PRIMARY KEY (conversation_id, seq))""")
client = anthropic.Anthropic()

def load(conv_id):
    rows = db.execute(
        "SELECT role, content FROM messages WHERE conversation_id = ? ORDER BY seq",
        (conv_id,),
    ).fetchall()
    return [{"role": r, "content": c} for r, c in rows]

def reply(conv_id, user_text):
    history = load(conv_id) + [{"role": "user", "content": user_text}]
    msg = client.messages.create(model="claude-haiku-5-5", max_tokens=600, messages=history)
    text = "".join(b.text for b in msg.content if b.type == "text")
    n = len(history)
    with db:
        db.execute("INSERT INTO messages VALUES (?,?,?,?)", (conv_id, n, "user", user_text))
        db.execute("INSERT INTO messages VALUES (?,?,?,?)", (conv_id, n + 1, "assistant", text))
    return text

Call reply("c1", "Hi") and then reply("c1", "What did I just say?") and the second answer refers to the first, even if you restart the program in between.

Per-user isolation

Keep one history per conversation and never share lists between users. A global list in a web server is a classic bug that leaks one user's data into another's answers.

Checklist

• History is stored per conversation ID.
• You append the assistant reply after every successful call.
• You have a plan for long conversations (trimming or summarising).
• You track tokens per turn so you can see growth.