Cosine Similarity

Cosine similarity measures how alike two vectors are in direction, ignoring their length: the dot product of the two vectors divided by the product of their magnitudes, producing a score from -1 (opposite) through 0 (unrelated) to 1 (identical direction).

Two embeddings can point the same way but have different lengths — one text is longer and its raw vector happens to have larger components — and cosine similarity is specifically built to not care. It measures the angle between vectors, not the distance between their tips, which is exactly the property embedding comparison needs: what matters is whether two pieces of text mean similar things, not which one produced a numerically larger vector.

How it works

For vectors A and B, cosine similarity is:

cos(A, B) = (A · B) / (||A|| × ||B||)

where A · B is the dot product (multiply corresponding components, sum the results) and ||A|| is the vector's magnitude (the square root of the sum of its squared components). The dot product alone would reward longer vectors unfairly; dividing by both magnitudes normalizes it into a pure measure of direction, bounded between -1 and 1 regardless of how large either vector's raw values are.

In practice, most embedding models output vectors that are already normalized to unit length, which collapses cosine similarity into a plain dot product — one reason it's the default similarity metric in nearly every vector database and embedding API.

Why it matters

Cosine similarity is the arithmetic underneath almost every claim of "these two things are similar" in modern AI systems: semantic search ranks by it, RAG retrieval ranks by it, deduplication and clustering threshold on it. Understanding that it is a bounded, direction-only measure — not a probability, not a percentage, and not comparable across different embedding models with different vector spaces — prevents a common mistake: treating a raw cosine score as an absolute confidence level rather than a relative ranking signal.

A worked example

Take two toy 4-dimensional vectors representing simplified feature weights: A = [4, 1, 0, 2] and B = [3, 2, 1, 1].

A · B = (4×3) + (1×2) + (0×1) + (2×1) = 12 + 2 + 0 + 2 = 16

||A|| = sqrt(16+1+0+4) = sqrt(21) ≈ 4.583

||B|| = sqrt(9+4+1+1) = sqrt(15) ≈ 3.873

cos(A, B) = 16 / (4.583 × 3.873) ≈ 0.901

A high similarity, consistent with the two vectors sharing weight on the same components. Compare A against C = [0, 0, 4, 0], which only has weight on the one component A has zero in: A · C = 0, so cos(A, C) = 0.0 exactly — orthogonal, meaning entirely unrelated by this measure, regardless of either vector's magnitude.

How miatz teaches it

The cosine similarity lab has learners compute the formula by hand on 3- and 4-dimensional toy vectors before ever calling an embedding API, so the number that comes back from a real model — 0.83, 0.41, whatever it is — is a value they know how to derive, not a mysterious score to trust blindly.