Build Your First Semantic Search in Python with NumPy

You can build a real semantic search with a short Python script: embed your documents, embed the query, compute similarity with one matrix multiplication, and sort. For a few thousand documents this brute-force approach is fast enough and needs no database. It is also the best way to understand what a vector database does for you later.

Setup

Install the two libraries with pip install sentence-transformers numpy. We use a small local model so the example runs without an API key.

The complete script

The script below indexes six short documents and answers queries. Notice that the query shares few words with the best document, which is the point of semantic search.

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

documents = [
    "Refunds are available within 30 days of purchase.",
    "Our API rate limit is 100 requests per minute.",
    "To change your password, open Settings and select Security.",
    "Invoices are emailed on the first day of each month.",
    "You can export your data as CSV from the Reports page.",
    "Two-factor authentication adds a second login step.",
]

# 1. Index: embed all documents once. Normalize so dot product = cosine.
doc_vectors = model.encode(documents, normalize_embeddings=True)

def search(query, k=3):
    q = model.encode([query], normalize_embeddings=True)[0]
    scores = doc_vectors @ q            # one score per document
    top = np.argsort(-scores)[:k]       # indexes of the best scores
    return [(float(scores[i]), documents[i]) for i in top]

for score, text in search("how do I get my money back?"):
    print(f"{score:.3f}  {text}")

Expected output is a ranked list in which the refund sentence is first with a clearly higher score than the rest. The exact numbers depend on the model, so do not rely on a specific value. What matters is the order.

How it works

1. model.encode(documents, normalize_embeddings=True) returns a matrix with one row per document. Normalizing makes every row have length 1.
2. At query time we embed the query the same way and get one vector.
3. doc_vectors @ q multiplies the matrix by the vector. Because the rows are normalized, each result is the cosine similarity between the query and one document.
4. np.argsort(-scores) sorts descending, and we keep the first k.

That is exact nearest neighbor search. It compares the query with every vector, so the cost grows linearly with the number of documents. With 100,000 vectors of 384 dimensions, a single query is a matrix-vector product of about 38 million multiplications, which a modern CPU handles quickly. Millions of vectors, many users or strict latency targets are where indexes and databases start to matter.

Make it reusable

Embedding is the slow part, so avoid repeating it. Save the matrix and the documents to disk once, and load them when the app starts.

import json
import numpy as np

np.save("doc_vectors.npy", doc_vectors)
with open("documents.json", "w", encoding="utf-8") as f:
    json.dump(documents, f)

# later, in your app
doc_vectors = np.load("doc_vectors.npy")
with open("documents.json", encoding="utf-8") as f:
    documents = json.load(f)

Try these experiments

• Search for "stay safe when logging in" and see which document wins.
• Search for an exact phrase such as "100 requests per minute" and compare the score with a paraphrase.
• Add a document that says the opposite of another (for example, "Refunds are not available") and watch how close it ranks. This shows that similarity is about topic, not truth.
• Search for something unrelated. Notice that the system still returns results, just with low scores. Real apps need a threshold or a "no answer" path.

Why normalizing helps

The line normalize_embeddings=True scales each vector to length 1. After that, a plain dot product is exactly cosine similarity, so the whole search is one matrix-vector multiplication that NumPy runs in optimized native code. If you forget to normalize, the scores become a mix of angle and length, and long or unusual vectors can win for the wrong reason. Normalize once at index time and once for the query, and the rest of the code stays simple.

Also notice that the index is just a NumPy array. Adding a document means encoding it and stacking a new row with np.vstack. Deleting one means removing a row and the matching text. This tiny design already has the two core operations every vector store offers: add vectors and find nearest vectors. A database adds persistence, indexes for speed, filters and concurrent access around the same idea.

Limits of this approach

• Documents must be short enough for the model's input limit, so real data needs chunking.
• Everything lives in memory and there is no filtering by metadata.
• Adding or deleting documents means editing the matrix by hand.
• There is no keyword matching, so rare identifiers can be missed.

Each limit maps to a later topic: chunking, vector databases, metadata filters and hybrid search. For now you have a working retriever, which is the core of RAG. In the next posts you will learn how to choose the embedding model and how to prepare real documents.