Database
Database
How vector databases power semantic search and RAG — pgvector vs. dedicated engines, HNSW vs. IVF indexing, and hybrid search.
A vector database stores high-dimensional embeddings and answers approximate-nearest-neighbour queries: given a query vector, return the most semantically similar stored vectors fast. They are the storage layer underneath almost every RAG system, semantic search feature, and recommendation engine shipping in 2026, and the category has consolidated from a 2023 land-grab into a few durable choices.
The defining decision for most teams is pgvector-on-Postgres versus a dedicated engine (Pinecone, Weaviate, Qdrant, Milvus). Notifire tracks the releases that move that line: pgvector's HNSW and quantization support, the dedicated engines' move toward hybrid (dense + sparse/BM25) retrieval, and the indexing trade-offs — HNSW's recall-versus-memory profile against IVF's faster build and smaller footprint — that determine cost at scale.
No briefings on this topic yet. Subscribe to the feed and we’ll cover it as the news cycle moves.
A database optimised for storing embeddings — numeric vectors that capture the meaning of text, images, or audio — and retrieving the closest matches to a query vector via approximate nearest-neighbour (ANN) search. It's the retrieval layer behind semantic search, RAG, and recommendation systems, where the goal is finding things similar in meaning rather than exact keyword matches.
If you already run Postgres and your workload is moderate — roughly under 100M vectors and under a few hundred QPS — pgvector with an HNSW index is usually the right call: one database, one backup, one operations team, plus transactional consistency with your relational data. Above that scale, or when you need very high QPS and advanced filtering, a dedicated engine like Qdrant, Milvus, or Pinecone starts to earn its keep.
HNSW (Hierarchical Navigable Small World) is a graph index with excellent recall and low query latency, but it builds slowly and holds the whole graph in memory. IVF (inverted file) clusters vectors and searches only the nearest clusters — faster to build and lighter on memory, at some cost to recall. HNSW is the default for most quality-sensitive workloads; IVF (often with product quantization) wins when memory or index-build time dominates.
Hybrid search combines dense vector similarity with sparse keyword scoring (BM25 or SPLADE), then fuses the rankings — commonly with reciprocal rank fusion. Pure vector search misses exact-match needs like product SKUs, error codes, or rare proper nouns, while keyword search misses paraphrase. Fusing both consistently beats either alone, which is why every major engine now ships hybrid retrieval as a first-class feature.
The Notifire briefing
Verified tech intelligence in your inbox — AI, security, infra, and data.