Step-by-step articles on Generative AI, part of our AI & Machine Learning collection.
Showing 20 articles
Large language model APIs all follow the same basic shape: you send a list of messages and get a message back. This guide uses the official Anthropic SDK to call Claude, but the ideas carry over to any provider. 1. Install the SDK and set your key # TypeScript...
An LLM only knows what was in its training data. Retrieval-augmented generation (RAG) lets it answer questions about your data, such as docs, tickets or policies, by finding the relevant passages first and putting them in the prompt. The pipeline Ingest: split...
Long answers can take many seconds to generate. Streaming shows text as it is produced, so users see progress immediately instead of staring at a spinner. Streaming on the server The SDK's messages.stream() helper gives you an async iterator of events plus a t...
An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meanings get vectors that point in similar directions, even if they share no words. from sentence_transformers import SentenceTransformer model = Se...
The system prompt is the brief you give the model before the conversation starts. Most poor LLM results come from a vague brief, not a weak model. 1. Give context, not just commands Explain who the users are and why the task matters. A model that understands t...
Chunking decides what the retriever can find. Chunks that are too small lose context; chunks that are too large mix several topics and dilute the embedding. 1. Fixed-size chunks with overlap The simplest approach: split every N tokens or characters and repeat...
Asking a model to "reply in JSON" works most of the time. Production code needs it to work every time. Structured outputs constrain the response to a JSON Schema, so the result always parses. TypeScript with Zod import Anthropic from '@anthropic-ai/sdk'; impor...
You don't always need a separate vector database. The pgvector extension adds a vector type, distance operators and approximate indexes to PostgreSQL, so embeddings live next to the rest of your data. Set up the table CREATE EXTENSION IF NOT EXISTS vector; CRE...
Many apps send the same large prefix on every request: a long system prompt, tool definitions, or a document the user is asking about. Prompt caching stores that prefix so later requests read it from cache at a fraction of the normal input price and with lower...
Vector search is great at meaning ("cancel my subscription" finds "how to end your plan") but weak at exact tokens: error codes, product SKUs, function names. Keyword search is the opposite. Hybrid search runs both and merges the results. Keyword search in Pos...
LLM APIs bill by the token , a chunk of text that is often a word or part of a word. Input and output tokens are priced separately, and output tokens cost several times more than input tokens. Count tokens before you send Token counts depend on the model's tok...
Embedding search compares a query vector with chunk vectors that were computed separately. It is fast, but it misses nuance. A reranker reads the query and each candidate chunk together and scores how well they match. It is slower, so you use it on a short lis...
LLM APIs are network services under heavy load, so failures are normal. The trick is knowing which errors to retry and which to surface to the user. Retryable vs non-retryable Retry: 429 rate limit, 5xx server errors, overloaded errors, timeouts and dropped co...
Users trust an answer more when they can click through to the source. Citations also make hallucinations easy to spot: an unsupported claim has nothing to point at. Option 1: Ask for chunk IDs Answer the question using only the sources below. After each senten...
Few-shot prompting means showing the model a handful of input/output examples before the real input. It is the fastest way to teach a specific style or labelling scheme without fine-tuning. A classification example Label each commit message as feat, fix, docs...
Without evaluation, every change to chunking, embeddings or prompts is a guess. A small labelled dataset turns RAG tuning into engineering. 1. Build an evaluation set Collect 50-200 real questions from logs, support tickets or users. For each, record the ID(s)...
Reasoning models can "think" before answering: they work through a problem internally, then write the final response. Thinking improves results on hard tasks such as multi-step maths, debugging and planning, at the cost of extra tokens and time. Adaptive think...
Exact nearest-neighbour search checks every vector, which gets slow past a few hundred thousand rows. Approximate nearest neighbour (ANN) indexes trade a little recall for a huge speed-up. pgvector offers two: HNSW and IVFFlat . HNSW (Hierarchical Navigable Sm...
Modern LLMs read images and PDFs as well as text. That makes it easy to build features like "extract the line items from this invoice" or "explain this error screenshot". Image from a URL const response = await client.messages.create({ model: 'claude-opus-5-5'...
A chunk like "Revenue grew 3% over the previous quarter" is useless on its own: which company, which quarter? Contextual retrieval , a technique published by Anthropic, fixes this by adding a short explanation of where the chunk sits in its document before emb...
