The relationship between Dot Product and Cosine Similarity on normalized vectors.
For normalized (unit-length) vectors, dot product and cosine similarity are identical: a · b = cos(a, b) when ‖a‖ = ‖b‖ = 1, because cos(a, b) = (a · b) / (‖a‖ ‖b‖) and the denominators are 1. So indexing with normalized vectors and ranking by (negative) dot product gives the same ordering as ranking by cosine distance.
Summary
- Vector DBs often implement “cosine” by storing normalized vectors and using inner product. Dot product is a single multiply-add loop—no division, no sqrt—and maps well to SIMD and GPU.
- Normalize the query the same way before each search; otherwise the equivalence breaks and recall can suffer.
- Pipeline: normalize at ingest and at query time; create collection with cosine or IP; the index then uses one loop of multiply-adds per comparison.
- For unnormalized vectors, cosine requires norm computation (extra cost); for normalized, cosine and dot product are the same operation.
- Same nearest-neighbor ordering: ranking by (negative) dot product on unit vectors equals ranking by cosine distance.
Why this equivalence matters
This equivalence is why many vector databases and ANN libraries implement “cosine” search by storing normalized vectors and using the inner product (or its negative as a distance). Dot product is a single loop of multiply-adds—no division, no square root—and maps well to SIMD and GPU kernels. So you get cosine semantics with the speed of inner product.
Trade-off: if you store unnormalized vectors and want cosine, you must compute norms at query time (or store them), which adds cost. Normalizing once at ingest and once per query avoids that and keeps a single, fast code path. Practical tip: verify that your embedding API output and your VDB’s “cosine” option both assume unit-length vectors so that scores are comparable.
Normalizing at ingest and at query time
If your embedding model does not output unit vectors, you can normalize at ingest time and then use dot product for search. The only caveat is that you must normalize the query vector the same way before each search.
Failing to normalize the query (or storing unnormalized and querying with normalized) breaks the equivalence and can hurt recall. Pipeline: run the same normalization (e.g. L2 unit length) on every document vector before upsert and on the query vector before each search; then the index’s dot-product comparisons are exactly cosine similarity.
Implementation and ANN indexes
ANN indexes that support inner product (e.g. HNSW with IP) work for cosine when vectors are normalized. No change to the index structure is needed—only the stored and query vectors must be unit length. See computation overhead of different distance metrics for cost comparison when norms are or aren’t precomputed.
Practical tip: when debugging “cosine” search, verify that both document and query vectors are unit length (e.g. check ‖v‖ ≈ 1.0). If the engine expects normalized input and you pass unnormalized queries, scores will be wrong and ranking can degrade. Normalized vs. unnormalized distance scores explains how score ranges and thresholding differ.
Frequently Asked Questions
Do I need to store normalized vectors?
For cosine-via-dot-product you do: store unit vectors and normalize the query; then dot product = cosine.
What if I forget to normalize the query?
Ranking changes: longer queries get higher dot products with long vectors. Always normalize the query when the index stores normalized vectors.
Is there a speed difference between cosine and dot product?
For normalized vectors, no—they’re the same computation. For unnormalized, cosine needs extra norm computation; computation overhead of different distance metrics compares the cost of each option.
Does this work with approximate search (ANN)?
Yes. ANN indexes that support inner product (e.g. HNSW with IP) work for cosine when vectors are normalized.