The concept of “Embedding” simplified

What an embedding is

An embedding is a fixed-size vector that stands for a piece of data—a sentence, a paragraph, an image, a user ID, etc. The crucial property: if two inputs are semantically or structurally similar, their embedding vectors are close under a distance like cosine similarity or Euclidean distance. You don’t hand-write embeddings; a model (e.g. a neural network) computes them. For text, models like transformers turn a passage into a fixed-size vector. For images, CNNs or vision transformers do the same. So you can store embeddings in a vector database and query by “vectors near this one” to get similar content.

The dimensionality of the embedding (e.g. 384, 768, 1536) is fixed by the model. Higher dimensions can capture more nuance but use more storage and compute; the right choice depends on your corpus size, recall requirements, and latency budget. Embeddings are typically dense (most dimensions non-zero), unlike sparse term-based representations used in classic IR.

Latent space and normalization

Embeddings are usually dense (most dimensions non-zero) and live in a latent space—a space learned by the model, where the dimensions aren’t hand-named but capture abstract features. Normalizing them (e.g. to unit length) is common so that similarity can be measured with cosine or dot product without magnitude dominating. In short: embedding = turning data into a vector so that similarity in the vector space mirrors similarity in the real world; that’s the bridge between raw unstructured data and fast, meaning-based search in a VDB.

The latent space is shared: all items embedded with the same model live in the same space, so you can compare text to text, image to image, or—with multimodal models—text to image. Unit-length normalization makes cosine similarity and dot product equivalent and stabilizes training and retrieval.

How embeddings are used in a VDB

In a typical flow, you embed your corpus (e.g. document chunks) and upsert the vectors into a collection with optional metadata. At query time you embed the query with the same model and run ANN or exact nearest-neighbor search. The choice of embedding model and its dimensionality affects recall, memory use, and latency. For domain-specific data, fine-tuning the model can improve relevance.

Index and query must use the same model and normalization so that distances are comparable. If you upgrade or change the model, you typically need to re-embed the corpus and rebuild the index. Metrics like recall@k and latency (p50, p99) help you evaluate whether an embedding model is right for your workload.

Cross-encoders vs. bi-encoders

Most VDB workflows use bi-encoders: query and documents are encoded separately into vectors, and similarity is computed between vectors (e.g. cosine). Cross-encoders take query and document together and output a single relevance score; they’re more accurate but too slow to run against every document. So the usual pattern is: bi-encoder for retrieval (vector DB), then optionally re-rank a small set with a cross-encoder. See cross-encoders vs. bi-encoders for more.

Bi-encoders scale because you encode each document once and each query once; similarity is just a distance between two vectors. Cross-encoders require the query and each candidate document to be processed together, so they’re used as a second stage on a small candidate set (e.g. top 20 or 100 from the vector DB) to improve precision without sacrificing speed.