Vector Databases Explained: What They Are and When You Need One

What a Vector Database Actually Stores

The simplest vector database explained version is this: instead of storing rows and columns like a normal database, it stores embeddings — long lists of numbers that represent the meaning of a piece of text, image, or other content — and it's built to answer one specific question extremely fast: which of these millions of embeddings is most similar to this new one. That similarity search is the entire point. A regular database can filter by exact matches or ranges efficiently; it's not built to answer "what's conceptually close to this," which is the query pattern that powers semantic search, recommendation, and RAG.

Why Regular Databases Can't Do This Job Well

You can technically store embeddings as arrays in Postgres and compute similarity in application code, and for small datasets that's genuinely fine — we've shipped that exact approach for clients with modest data volumes, because adding a whole new database system for ten thousand records is unnecessary complexity. But once you're searching across millions of vectors, doing that comparison naively means comparing the query against every single stored vector, which doesn't scale. Purpose-built vector databases use approximate nearest-neighbor indexing to make that search fast at scale, trading a small amount of accuracy for a massive speed gain — and that trade-off is usually invisible to end users.

When You Actually Need One

The honest answer is: later than most teams think. If you're building a RAG system, a vector store is core infrastructure, no way around it. But if you're prototyping, or your document set is small enough to fit comfortably in memory, extensions like pgvector inside a database you already run handle the job without adding an entirely new system to your infrastructure, your monitoring, and your team's operational knowledge. We default new clients to pgvector unless there's a clear, specific signal — scale, query latency requirements, or a need for advanced filtering the extension doesn't support — that justifies a dedicated vector database like Pinecone, Weaviate, or Qdrant.

What Actually Differs Between Vector Database Options

The differences that matter in practice are less about raw search speed, since most options perform adequately for typical workloads, and more about operational fit: does it support the metadata filtering your queries actually need, how does it handle updates and deletes at your expected write volume, and does it run where the rest of your infrastructure already lives or does it add a new vendor relationship and a new thing to monitor. We've picked different vector databases for different clients based almost entirely on these operational fit questions, not benchmark numbers.

A vector database doesn't make your search smarter. It makes 'smarter search' fast enough to actually ship. Conflating those two problems leads teams to adopt one before they need it.

The Embedding Model Matters More Than the Database

Teams spend a lot of energy choosing between vector database vendors and comparatively little choosing an embedding model, even though the embedding model determines what "similar" even means for your data. A general-purpose embedding model trained mostly on web text will perform noticeably worse on specialized domain content — legal documents, medical records, internal jargon-heavy company data — than one fine-tuned or selected for that domain. We test embedding model choice against real client documents before finalizing any vector infrastructure decision, because the best vector database in the world can't fix embeddings that don't represent your content's actual meaning well.

Start Simple, Scale the Infrastructure Later

Our default advice, having explained vector databases to a lot of clients who assumed they needed enterprise infrastructure from day one, is to start with the simplest option that works — pgvector, a managed extension, or even an in-memory library for a prototype — and treat migrating to a dedicated vector database as a scaling decision you make once you have real usage data showing you need it. That migration is usually straightforward when the time comes. Building unnecessary infrastructure complexity into an early-stage product rarely is.

Hybrid Search Often Beats Pure Vector Search

A vector database explained purely as a semantic-similarity engine misses a practical reality: pure vector search sometimes loses to a hybrid approach that combines vector similarity with traditional keyword search, especially for queries involving exact terms like product codes, names, or numbers that embeddings don't represent precisely. Most mature vector databases now support this hybrid mode natively. We test both approaches against real client queries before committing to one, because the theoretically more sophisticated option — pure semantic search — sometimes performs worse in practice than a well-tuned hybrid of the two.

Migration between vector databases, when it eventually happens, is usually less painful than teams fear, provided the embedding pipeline was built with portability in mind from the start. We keep the embedding generation step decoupled from the specific vector store client — a thin interface in front of whichever database is in use — so swapping pgvector for a dedicated vector database later means changing a storage backend, not re-architecting the whole retrieval pipeline. That one design decision, made early and cheaply, is what makes 'start simple and scale later' an actually realistic strategy instead of just reassuring advice.

If you're scoping something like this, see our AI Studio.