Dense Retrieval vs. Sparse Retrieval

Dense retrieval finds relevant documents by comparing the meaning of a query against the meaning of every document, using learned vector embeddings and nearest-neighbor search. Sparse retrieval finds relevant documents by comparing the words a query and a document have in common, using weighted term-matching algorithms like BM25 over an inverted index. Both are old ideas wearing different-generation clothes: sparse retrieval descends from classical information retrieval going back to the 1970s, dense retrieval from neural encoders that only became practical for large-scale search around 2020. The choice between them, and increasingly how to combine them rather than choose, is one of the first architectural decisions behind every modern search engine, RAG pipeline, and enterprise knowledge base.

Sparse Retrieval: Matching Words

Sparse retrieval represents a query or document as a vector where almost every dimension is zero: one dimension per term in the vocabulary, with a nonzero weight only where that term actually appears. This sparsity is what gives the family its name, and it maps directly onto the inverted index, a data structure that stores, for every term, the list of documents containing it. Looking up matches for a query becomes a fast lookup and merge over a handful of posting lists instead of a scan over the full corpus.

BM25 (Best Matching 25) is the standard scoring function for sparse retrieval and remains, decades after its introduction, the default baseline that every new retrieval method is measured against:

score(D, Q) = Σ IDF(qᵢ) · (f(qᵢ, D) · (k₁ + 1)) / (f(qᵢ, D) + k₁ · (1 − b + b · |D| / avgdl))

Each query term contributes a score based on how rare it is across the corpus (inverse document frequency), how often it appears in this document (term frequency, with diminishing returns via the k₁ saturation constant), and a length-normalization term (b) that prevents long documents from winning purely by containing more words. No training is required: BM25 is a closed-form statistical formula, computed directly from term counts.

This gives sparse retrieval real strengths: it is exact, interpretable (you can point to precisely which shared terms drove a match), cheap to run at any scale, and unbeatable at matching rare, specific tokens such as product SKUs, error codes, legal citations, or a person’s name, exactly the tokens a dense model tends to blur together. Its weakness is the mirror image of that strength: it only sees vocabulary, not concepts. A query about a “heart attack” will not match a document that only says “myocardial infarction,” even though they mean the same thing. This is the classic vocabulary mismatch problem.

Dense Retrieval: Matching Meaning

Dense retrieval replaces the sparse, mostly-zero term vector with a dense, low-dimensional embedding: every document is passed once through a neural encoder and reduced to a few hundred or thousand real-valued numbers that capture its meaning, not its exact wording. Queries are encoded with the same (or a paired) model at search time, and retrieval becomes nearest-neighbor search in that embedding space, typically accelerated with an index like HNSW.

Because the query and document are each encoded independently, with no interaction between them until the similarity score is computed, dense retrievers are architecturally bi-encoders; see Cross-Encoder vs. Bi-Encoder for how that compares to the joint-encoding models used downstream for reranking. Dense retrieval only became competitive with decades-tuned sparse baselines once Dense Passage Retrieval (DPR) showed, in 2020, that a BERT-based bi-encoder trained with contrastive learning on question-passage pairs could outperform a strong Lucene-BM25 system by 9 to 19 points of top-20 retrieval accuracy on open-domain QA. That result is generally treated as the moment dense retrieval became the default choice for semantic search rather than a research curiosity.

Dense retrieval’s strength is exactly sparse retrieval’s weakness: it generalizes across paraphrase, synonymy, and cross-lingual phrasing, because the encoder has learned what concepts mean, not just which characters appear. Its weakness is the mirror image too: embeddings compress meaning into a fixed-size vector, and fine-grained lexical detail, an exact part number, a specific date, a rare acronym the encoder never saw much of during training, can get lost in that compression. This is sometimes called the exact-match problem, and it’s why a purely dense system can confidently retrieve a plausible-sounding but factually wrong passage when the query hinges on one precise token.

Architecture Side by Side

graph TB
    classDef default fill:#ffffff,stroke:#4338CA,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef data    fill:#EEF0F7,stroke:#0D9488,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef process fill:#F7F8FC,stroke:#6366F1,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef output  fill:#4338CA,stroke:#4338CA,stroke-width:2px,color:#ffffff,rx:8px,ry:8px;

    subgraph SP["Sparse Retrieval"]
        D1([Documents]):::data --> TOK[Tokenize + Weight<br/>TF-IDF / BM25]:::process --> INV[(Inverted Index)]:::data
        Q1([Query]):::data --> QTOK[Tokenize]:::process --> INV
        INV --> SR[Term-Overlap Ranking]:::output
    end

    subgraph DN["Dense Retrieval"]
        D2([Documents]):::data --> ENC1[Embedding Model]:::process --> VEC[(Vector Index HNSW)]:::data
        Q2([Query]):::data --> ENC2[Embedding Model]:::process --> VEC
        VEC --> DR[Similarity Ranking]:::output
    end

Both pipelines share the same shape: an offline indexing path that processes documents once, and an online query path that only has to process the incoming query, which is what makes either approach fast enough for production search at scale.

Head-to-Head Comparison

Learned Sparse Retrieval: A Middle Ground

SPLADE (Sparse Lexical and Expansion model) sits between the two families. Like classical sparse retrieval, it produces a sparse, vocabulary-sized vector that plugs directly into an ordinary inverted index, keeping the speed and interpretability of BM25-style search. But unlike BM25, the weights aren’t hand-derived statistics: a transformer learns which terms to weight highly, and which related terms to add even if they never appeared in the original text (query and document expansion), closing much of the vocabulary-mismatch gap that plain sparse retrieval suffers from. SPLADE and its successors (SPLADE v2, and 2025-era variants) are the clearest evidence that “sparse” and “semantic” are not opposites: sparsity is a representation choice, and semantics can be learned on top of it.

Hybrid Retrieval: Combining Both

Because dense and sparse retrieval fail in complementary ways, production systems increasingly run both and fuse the results rather than committing to one. Two fusion strategies dominate:

• Score-based fusion (e.g. Weaviate’s alpha parameter): normalize and linearly combine the BM25 score and the vector similarity score, with a tunable weight controlling how much each contributes.
• Rank-based fusion, most commonly Reciprocal Rank Fusion (RRF): ignore the raw scores (which live on incomparable scales) and combine results using only each item’s rank position in each list, as shown in the code example above.

Some embedding models now collapse this into a single pass: BGE-M3 produces dense, sparse, and multi-vector (ColBERT-style) representations from one forward pass, so a hybrid pipeline doesn’t require hosting two separate models.

What’s New (2025-2026)

• Hybrid search as the default, not the advanced option. Qdrant’s Universal Query API (introduced in Qdrant 1.10 and matured through 2025) lets a single query combine dense vectors, sparse vectors, and even ColBERT-style multi-vectors with a chosen fusion method in one call, reflecting a broader shift: hybrid retrieval is now the out-of-the-box recommendation from most major vector databases (Qdrant, Weaviate, Elasticsearch, Pinecone), not a manual pipeline teams had to assemble themselves.
• Unified dense/sparse/multi-vector models. BGE-M3-style models, which emit all three representations from one encoder, have become the practical default for teams that don’t want to operate a separate BM25 stack and a separate embedding model.
• Learned sparse retrieval in production. SPLADE-family models moved from research benchmarks to production search in 2025-2026, particularly in e-commerce and enterprise search where exact product identifiers and codes matter alongside semantic intent.
• Reranking as the standard third stage. Whether retrieval is dense, sparse, or hybrid, teams increasingly treat the retrieved candidate set as an input to a separate reranking stage rather than a final answer, since neither dense nor sparse (nor their fusion) fully replaces the precision of a joint query-document model.

Practical Guidance

Neither family is strictly better: sparse retrieval is a precise, cheap, statistically grounded baseline that can’t see past vocabulary, and dense retrieval understands meaning at the cost of losing exact lexical precision. The dominant pattern in 2025-2026 production systems isn’t choosing one, it’s routing both into a shared candidate set and letting fusion, and often a reranker downstream, sort out which evidence actually answers the query.

from rank_bm25 import BM25Okapi
from sentence_transformers import SentenceTransformer
import numpy as np

corpus = [
    "Qdrant's Universal Query API fuses dense and sparse vectors in one call.",
    "BM25 ranks documents by term frequency, inverse document frequency, and length.",
    "SPLADE learns sparse term weights and expansions directly from a transformer.",
    "Restarting the Kubernetes pod cleared the OOMKilled error on node-4.",
]

# Sparse side: BM25 over a tokenized corpus (exact term matching)
tokenized_corpus = [doc.lower().split() for doc in corpus]
bm25 = BM25Okapi(tokenized_corpus)

# Dense side: sentence embeddings (semantic matching)
encoder = SentenceTransformer("BAAI/bge-base-en-v1.5")
doc_vectors = encoder.encode(corpus, normalize_embeddings=True)

query = "How does BM25 score a document?"

sparse_scores = bm25.get_scores(query.lower().split())
sparse_rank = np.argsort(sparse_scores)[::-1]

query_vector = encoder.encode(query, normalize_embeddings=True)
dense_scores = doc_vectors @ query_vector
dense_rank = np.argsort(dense_scores)[::-1]

# Fuse both rankings with Reciprocal Rank Fusion
# see /glossary/reciprocal-rank-fusion-rrf
def rrf_fuse(*rankings, k=60):
    fused = {}
    for ranking in rankings:
        for pos, idx in enumerate(ranking):
            fused[idx] = fused.get(idx, 0) + 1 / (k + pos + 1)
    return sorted(fused, key=fused.get, reverse=True)

for idx in rrf_fuse(sparse_rank, dense_rank):
    print(corpus[idx])