Text Embeddings Explained: Turning Meaning into Vectors

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meanings get vectors that point in similar directions, even if they share no words.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
vecs = model.encode([
    "How do I reset my password?",
    "I forgot my login credentials",
    "What's the weather in Mumbai?",
], normalize_embeddings=True)

print(vecs.shape)          # (3, 384) -> three 384-dimensional vectors
print(vecs[0] @ vecs[1])   # high similarity: same intent, different words
print(vecs[0] @ vecs[2])   # low similarity

Cosine similarity

Similarity is usually measured by the angle between vectors. Cosine similarity is 1 for identical direction and around 0 for unrelated text. If vectors are normalised to length 1, cosine similarity is just the dot product, which is fast to compute.

import numpy as np

def cosine(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

Choosing an embedding model

• Hosted APIs (for example Voyage AI, OpenAI, Cohere, Google) - strong quality, no GPUs to run, pay per token.
• Open-source models (sentence-transformers, BGE, E5, Nomic) - run locally or on your own servers, data never leaves your infrastructure.
• Check the model's maximum input length; text beyond it is truncated.
• Higher dimensions cost more storage and slower search; many models let you shorten vectors with little quality loss.

Rules that save you pain

• Use the same model for documents and queries. Vectors from different models are not comparable.
• Store the model name and version with each vector so you know what to re-embed after an upgrade.
• Some models expect different prefixes or input types for queries and documents; follow the model card.
• Benchmark on your own data. Public leaderboards don't always reflect your domain.