An embedding model turns text into a list of numbers — a point in a high-dimensional space. Retrieval decides which points are "close" to your query. The usual answer is one number: the cosine of the angle between two vectors. Here's what that actually means.
Feed a sentence into an embedding model — text-embedding-3-small,
all-MiniLM-L6-v2, whichever — and it hands back a fixed-length array of floats.
OpenAI's small model returns 1536 of them; a compact sentence-transformer model might return 384.
The individual coordinates have no assigned meaning — the model just
learned, during training, to place semantically similar text near each other in that space, and
dissimilar text far apart.
You can't easily draw 1536 dimensions. But the comparison math only cares about the angle between two vectors, and angles are angles no matter how many dimensions you're in — so a 2D picture is an honest simplification, not a cheat.
Drag the slider. Vector A is fixed — call
it your query, "PermissionDenied: Requires read access".
Vector B is some document, rotating around it.
Notice what moved and what didn't: dragging the slider only ever changes the angle. Cosine similarity is blind to how long the vectors are — only their direction counts. That's deliberate: a short document and a long document about the same topic can produce embeddings of different magnitude, and you don't want retrieval penalizing length. Direction is where the meaning lives; magnitude varies for reasons that have little to do with topic.
You never measure the angle directly — there's no protractor in 1536 dimensions. Instead you compute it from two much simpler operations:
The numerator, A · B, is the dot product — multiply each pair of
matching coordinates and add them up: a₁b₁ + a₂b₂ + … + aₙbₙ. Geometrically, it's how
much of B points in the same direction as A.
The denominator, ‖A‖ ‖B‖, is just each vector's own length —
√(a₁² + a₂² + … + aₙ²) — multiplied together. Dividing by it is what cancels out
magnitude and leaves you with pure direction: a number between −1 (opposite) and
1 (identical direction), with 0 meaning unrelated.
One practical shortcut: if every vector is normalized to length
1 ahead of time (many embedding APIs do this — OpenAI's does — or make it trivial), the denominator is always
1 × 1 — so cosine similarity collapses into a plain dot product. That's why vector
databases often index and rank on raw dot product: it's the same ranking, with the division
already paid for upfront.
Now put four real candidate documents around that same query vector — angles chosen to match how a small embedding model would plausibly place them, closest meaning first:
Look how close permissions.md and security.md sit to each other above —
12° and 33° apart from the query, both well inside "clearly relevant." A pure cosine ranking has
no mechanism to know that only one of them directly documents
that exact error string; both simply look like nearby points. That's the same ambiguity that kept
showing up when we ran an agent against real Deno docs for this exact query — permissions.md
and security.md traded places across several runs. Vector similarity gets you into the
right neighborhood reliably; distinguishing the right house on the block is a separate problem.
No — and this trips people up. There's no universal "the" embedding space. Each model defines its
own: OpenAI's text-embedding-3 family, Cohere's embed-v4, a
sentence-transformers model like all-MiniLM-L6-v2 — each was trained separately and
produces vectors that live in that model's geometry alone.
A vector from one model compared against a vector from another isn't directly comparable — same shape of
array, different coordinate systems (they can be translated between models, which is itself proof they were never interchangeable as-is). Whatever model embeds your documents at index time has to be
the exact same model that embeds the query at search time.
Cosine similarity is one step in a pipeline, not the whole thing:
At real scale you stop computing cosine similarity against every document one at a time — that's linear in corpus size. Production vector stores build an approximate-nearest-neighbor index over the vectors first (HNSW, IVF, and similar structures), trading a little accuracy for the ability to search millions of vectors in milliseconds. That index is exactly the kind of extra layer that gets bolted on top of the raw math once the naive version stops scaling — cosine similarity is the idea; everything past it is infrastructure built to keep computing it fast.
1. When comparing two text embeddings, why does cosine similarity look at the angle between vectors rather than the distance between their endpoints?
Two vectors can point the same direction (same meaning) while having very different lengths — cosine similarity ignores that length difference on purpose.
2. How is cos(θ) actually derived from two vectors A and B?
Once both vectors are normalized to length 1, the denominator becomes 1 and cosine similarity collapses to a plain dot product — the trick every eval script in this repo relies on.
3. Why is cosine similarity specifically the thing retrieval systems rank documents by?
The bound (−1 to 1) and clean geometric meaning are exactly what make it easy to reason about and cheap to compute at scale.
4. Is the vector format produced by one embedding model directly comparable to vectors from a different embedding model?
Two 384-dimensional vectors from different models can share a shape but mean different things along each axis — mixing them in one index is a real, easy-to-make mistake.
5. How does a vector's cosine-ranked position relate to what ends up in an LLM's context window during retrieval-augmented generation?
This is the whole point of the ranking pass in the diagram above: cos(θ) decides which handful of documents actually make it into the prompt.