Vector Database

A vector database is a database purpose-built to store embedding vectors and to find the vectors nearest to a given query vector — fast, at scale, and without scanning every row.

A relational database is superb at exact lookups: find the row where user_id = 4471. It has no native notion of "find the rows most similar in meaning to this one," because similarity between two embeddings is a distance computed at query time, not a value you can index with a B-tree. Vector databases exist to make that computation cheap even across billions of vectors.

How it works

At minimum, a vector database stores each vector alongside metadata (source document, chunk id, access permissions) and supports a nearest-neighbor query: given vector Q, return the k closest stored vectors by cosine similarity, dot product, or Euclidean distance. The naive approach — compare Q against every stored vector — is exact but scales linearly with the collection: comparing against a million 768-dimensional vectors means roughly 768 million multiply-add operations per single query, before you've served a single user.

Production vector databases instead build an approximate nearest-neighbor (ANN) index — most commonly HNSW — trading off a small amount of exactness for a large speedup. They also handle the operational load a raw index cannot: filtering by metadata alongside similarity ("similar chunks, but only from documents this user can see"), incremental inserts and deletes without rebuilding the whole index, and persistence and replication.

Why it matters

Every RAG system, semantic search feature, recommendation engine, and deduplication pipeline needs somewhere to put its embeddings and a fast way to query them — that's the vector database's entire job. Pick the wrong one and you inherit either unbounded latency (brute force at scale), missing production features (no metadata filtering, no updates), or unnecessary cost (a specialized service where dedicated infrastructure would do at a fraction of the price). Most teams today reach for either a dedicated vector database (Pinecone, Qdrant, Weaviate) or a vector extension bolted onto an existing store (pgvector on Postgres) — the right choice depends on scale and how tightly the vectors need to live next to the rest of the application's data.

A worked example

A knowledge base of 1,000,000 chunks, embedded at 768 dimensions in 32-bit floats, needs 1,000,000 × 768 × 4 bytes = 3.072 billion bytes — about 2.86 GiB — just for the raw vectors, before indexes or metadata. Querying that collection by brute force costs roughly 768 million multiply-adds per search. An HNSW index run at a typical ef_search of 100 instead visits on the order of a few thousand candidate nodes per query — often two to three orders of magnitude fewer comparisons — trading a small amount of recall for that speedup.

How miatz teaches it

The vector database lab has learners load the same corpus into a brute-force flat index and an ANN index, then measure both query latency and recall side by side as the collection grows from thousands to millions of rows — so the trade-off is a chart they produced, not a claim they read.