Hybrid Search 2026: BM25 + Vector Search, RRF & Reranking

Most search systems fail when they force one retrieval method to solve every query.

Keyword search is excellent when the user types an exact product code, error message, API symbol, legal clause, or rare technical term. Semantic search is better when the user describes an idea using different words from the document. Real users do both.

That is why modern production search increasingly uses hybrid retrieval: run lexical and semantic retrieval together, combine their candidate sets, then rank the merged results.

The useful mental model is:

query
  │
  ├── lexical retrieval (BM25 / full text)
  │
  └── semantic retrieval (dense or sparse vectors)
          │
          ▼
      rank fusion
          │
          ▼
      filtered candidates
          │
          ▼
        reranker
          │
          ▼
      final results

Hybrid search is not simply “BM25 + embeddings.” The hard engineering questions are how you generate candidates, how you combine incompatible scores, where filters are applied, how many results reach the reranker, and how you prove that the new pipeline actually improves user outcomes.

Why lexical search is still essential

Vector search did not make keyword search obsolete.

Lexical retrieval remains extremely strong for queries containing:

error codes
SKU/product IDs
function names
API endpoints
version numbers
company names
rare domain terminology
quoted phrases

Suppose a developer searches:

ERR_CONNECTION_RESET nginx 499

Those exact tokens are highly informative. A semantic model may understand the general networking topic, but ordinary term matching can identify documents containing the precise error signature.

BM25 remains one of the most common lexical ranking algorithms. It rewards documents that contain the query terms while accounting for factors such as term frequency and document length.

For exact technical search, that behavior is often exactly what you want.

Why semantic search is still essential

Lexical search has the opposite weakness: the user and document may describe the same concept using different words.

Query:

How do I prevent the same payment from being charged twice after retries?

Relevant document:

Use idempotency keys for retry-safe payment mutations.

The semantic relationship is strong even though many words differ.

Dense embedding models convert text into vectors representing meaning. Approximate nearest-neighbor search then finds documents whose embeddings are close to the query embedding.

Semantic retrieval is especially helpful for:

paraphrases
natural-language questions
conceptual similarity
noisy user wording
cross-lingual or synonym-heavy retrieval

The problem is that vector similarity alone may rank conceptually related but operationally useless results above exact matches.

That is why hybrid search exists.

Sparse semantic retrieval belongs in the conversation too

“Semantic search” does not always mean a dense vector with hundreds or thousands of floating-point dimensions.

Sparse retrieval models produce weighted token-like representations. They can capture semantic expansion while remaining closer to lexical search behavior.

A modern hybrid system may combine:

BM25
+ dense vectors

or:

BM25
+ sparse semantic retrieval

or even:

BM25
+ sparse semantic
+ dense semantic

The more branches you add, however, the more important fusion and evaluation become.

Do not add three retrievers because a diagram looks sophisticated. Add a retrieval signal only if it improves real queries.

The score-scale problem

A common mistake is to combine BM25 and vector similarity scores directly.

For example:

BM25 score:          13.7
cosine similarity:    0.82

Those numbers are not naturally comparable.

A naive calculation such as:

0.5 * BM25 + 0.5 * cosine

gives the BM25 scale disproportionate influence.

This is why production hybrid systems use either:

rank-based fusion, or
score normalization followed by weighted combination.

Reciprocal Rank Fusion (RRF)

RRF combines result positions instead of raw scores.

Conceptually, if a document ranks highly in multiple retrieval lists, its fused score improves.

A simplified formula is:

RRF(document) = Σ 1 / (k + rank_i(document))

where rank_i is the document's position in each retrieval list and k is a rank constant.

The key advantage is that the underlying score systems do not need to be calibrated to one another.

Elastic currently recommends RRF as the default approach for hybrid search, and OpenSearch also exposes rank-based RRF through its hybrid-search pipeline.

Why RRF is a strong default

RRF is attractive because it is:

simple
robust across different score distributions
easy to reason about
less sensitive to embedding/BM25 score calibration

If your lexical and vector scores come from totally different systems, RRF is often the cleanest first implementation.

Score normalization and weighted fusion

Sometimes rank position throws away useful information.

Imagine two semantic results:

Result A cosine similarity = 0.93
Result B cosine similarity = 0.62

If they appear at ranks 1 and 2, RRF mostly cares about those positions—not the size of the difference.

OpenSearch's normalization processor exists for cases where the magnitude of score differences matters. It normalizes scores from multiple query clauses onto comparable scales and then combines them.

A weighted pipeline might conceptually look like:

normalized_BM25 * 0.4
+
normalized_semantic * 0.6

This is more tunable than RRF, but also easier to overfit.

Use weighted fusion when your relevance data demonstrates that the extra calibration improves search quality.

Do not start with eight arbitrary weights copied from someone else's benchmark.

Candidate depth matters

Hybrid retrieval usually works in stages.

Example:

BM25 top 100
+
vector top 100
      ↓
fusion
      ↓
top 50
      ↓
reranker
      ↓
top 10

If each first-stage retriever only returns five results, the correct result may never reach the fusion or reranking stage.

If each retriever returns 5,000 results, latency and reranking cost may explode.

You need to tune candidate depth against recall and latency.

Useful questions include:

At what k does recall stop improving materially?
How expensive is reranking each additional candidate?
How often does the correct document appear only in one branch?

Filters must be part of retrieval—not a cleanup step

Production search rarely runs over one public corpus.

Real systems have filters such as:

tenant_id
user permissions
language
product
region
document status
updated_at
content type

For private SaaS/RAG systems, authorization filters are non-negotiable.

Bad architecture:

retrieve globally
→ rerank
→ remove documents user cannot access

This can damage both security and relevance.

Correct architecture:

authenticated access scope
→ lexical retrieval within scope
→ semantic retrieval within scope
→ fusion

The model should never receive unauthorized candidates.

Filtering vector search has performance implications

Approximate vector indexes are optimized around nearest-neighbor traversal.

Strong metadata filters can change the candidate population dramatically.

For example:

10 million global vectors
→ tenant filter leaves 2,000 eligible vectors

The behavior of an ANN index under such filters may be different from an unfiltered benchmark.

Production evaluation should therefore include the same filters real users will use.

Do not measure recall on a global test index and assume the same result under tenant, language, or product constraints.

Reranking is where expensive relevance models belong

First-stage retrieval should be fast and broad.

A reranker can be slower because it only sees a bounded candidate set.

Typical pipeline:

fast retrieval → 30–100 candidates
             ↓
semantic/cross-encoder reranker
             ↓
5–20 final results

Elastic's current ranking documentation explicitly treats reranking as a second-stage operation over candidates generated by first-stage retrieval.

This separation is important.

Do not run an expensive LLM relevance judgment over your entire corpus.

Cross-encoder vs LLM reranking

Two common reranking strategies are:

Cross-encoder / dedicated reranker

The model scores query-document pairs directly.

Advantages:

relatively fast
purpose-built relevance scoring
easier to run over tens of candidates

LLM reranking

An LLM judges relevance using richer instructions and context.

Advantages:

flexible domain criteria
can reason about nuanced relevance

Trade-offs:

expensive
slower
nondeterministic

Use an LLM reranker only when the quality gain justifies the cost.

Hybrid search is especially useful for RAG

RAG systems need high recall before the LLM can generate a grounded answer.

The generation model cannot recover evidence it never receives.

A robust RAG retrieval pipeline may look like:

question
  │
  ├── BM25 exact-term retrieval
  ├── dense semantic retrieval
  │
  ▼
RRF fusion
  │
  ▼
ACL / metadata-safe candidate set
  │
  ▼
reranking
  │
  ▼
dedupe / parent-section expansion
  │
  ▼
LLM context

This is usually stronger than “top 5 nearest embeddings.”

Exact terms are a major reason hybrid beats vector-only RAG

RAG queries frequently contain identifiers:

INV-204
PostgreSQL error 23505
Kubernetes CrashLoopBackOff
v1.27.3
customer plan Pro-Annual

Semantic embeddings may treat these terms weakly.

Lexical retrieval is excellent at finding them.

That is why hybrid search often improves technical support, code/documentation search, legal retrieval, and enterprise knowledge systems.

Query rewriting should be optional, not mandatory

A model can transform a conversational query into a better search query.

Example:

User: "What happened after yesterday's deploy?"

Conversation context indicates:

checkout API latency after deploy 2026-09-16

A rewritten query can improve both lexical and semantic retrieval.

But query rewriting adds:

latency
cost
another model failure point

Use it only when evals show a meaningful retrieval improvement.

Multi-query retrieval can help ambiguous requests

Some queries contain several interpretations.

A system can generate multiple search formulations:

original query
+ exact technical query
+ broader semantic query

then fuse the result sets.

This can improve recall for complex questions, but it should be bounded.

Ten generated queries multiplied by two retrievers can create twenty search operations for one user request.

Track the marginal quality gain.

Deduplicate before the model sees context

Hybrid retrieval can return the same document or overlapping chunks from several branches.

Without deduplication, an LLM context may look like:

chunk A paragraph 1–4
chunk B paragraph 2–5
chunk C paragraph 3–6

The model sees repetition instead of more evidence.

Deduplicate using stable document/chunk IDs and overlap metadata.

A useful pattern is:

retrieve precise chunk
→ identify parent section
→ include one coherent parent context

This gives precise search with readable evidence.

Freshness should be an explicit ranking signal

Semantic relevance alone can surface obsolete content.

For domains with changing documentation, consider:

status = active
version = current
published_at / updated_at

as filters or ranking features.

An outdated policy that perfectly matches the query is still the wrong answer.

Search relevance includes authority and freshness, not just similarity.

Search evaluation needs labeled queries

Do not judge hybrid search by clicking around a demo and saying it “looks better.”

Build a query set from real usage.

For each query, record relevant documents or graded relevance levels.

Then measure metrics such as:

Recall@k

Did the candidate set contain a relevant result?

Precision@k

How many returned results were relevant?

MRR

How high was the first relevant result?

nDCG

How good was the ranking when documents have different relevance grades?

For RAG, retrieval recall is especially important: if the correct evidence is absent, generation cannot fix the problem.

Evaluate lexical, semantic, hybrid, and reranked pipelines separately

A useful experiment table might look like:

Pipeline	Recall@20	nDCG@10	p95 latency
BM25 only	...	...	...
Dense only	...	...	...
BM25 + dense + RRF	...	...	...
Hybrid + reranker	...	...	...

This tells you whether each layer earns its complexity.

If BM25 already dominates your catalog search, semantic infrastructure may not be worth operating.

Segment evaluation by query type

Averages can hide important failures.

Create categories such as:

exact identifier
rare technical term
natural-language question
synonym/paraphrase
long query
misspelled query
multi-concept query

You may discover:

BM25 wins exact IDs
semantic wins paraphrases
hybrid wins overall

That is useful architecture evidence.

Search latency must be measured end-to-end

Hybrid search can add parallel retrieval branches and reranking.

Measure:

lexical search latency
vector search latency
fusion latency
rerank latency
total p50/p95/p99

Run lexical and semantic branches in parallel where the system allows it.

The final user experience cares about total latency, not how elegant the retrieval stack looks.

Cache carefully

Search results can sometimes be cached for repeated public queries.

Private or rapidly changing search requires caution.

Cache keys may need to include:

tenant/access scope
query
filters
index version

Never allow a shared search cache to return one tenant's private results to another.

Embedding migrations need versioning

Dense retrieval depends on an embedding model.

When the model changes, query and document vectors must remain compatible.

Track:

embedding_model
embedding_version
dimension
index_version

A safe migration looks like:

v1 vectors continue serving
      ↓
build v2 vectors in parallel
      ↓
run same relevance evals
      ↓
switch traffic
      ↓
retire v1

Do not overwrite your only index before validating the new embedding model.

When hybrid search is unnecessary

Hybrid search is not automatically superior enough to justify extra infrastructure.

Keyword-only may be sufficient when:

queries contain exact structured terminology
the corpus is small
lexical relevance is already excellent

Semantic-only may be sufficient when:

exact identifiers barely matter
the corpus is highly natural-language
evaluation shows lexical retrieval adds little

The correct architecture is the simplest retrieval system that meets your relevance targets.

A production implementation path

A sensible rollout is:

1. Establish BM25 baseline
2. Build a labeled query/relevance set
3. Add semantic retrieval
4. Compare lexical vs semantic failures
5. Add RRF hybrid fusion
6. Tune candidate depths
7. Apply production filters/ACLs
8. Add reranking only if needed
9. Track latency + relevance together
10. Turn production misses into regression queries

This is much safer than building a complicated retrieval stack before you have a baseline.

Production checklist

Before shipping hybrid search, verify:

Final takeaway

Hybrid search works because lexical and semantic retrieval fail differently.

BM25 gives you exact-token precision. Dense or sparse semantic retrieval gives you meaning-based recall. Rank fusion combines their candidate sets without pretending their native scores are identical. Reranking lets a more expensive model focus only on the highest-value candidates.

The strongest production pattern is usually:

retrieve broadly
→ fuse robustly
→ filter securely
→ rerank narrowly
→ evaluate continuously

Do not replace search engineering with embeddings.

Build a relevance pipeline that uses each retrieval method for what it is actually good at.

Hybrid Search in 2026: BM25 + Vector Search, RRF, Reranking, and Production Relevance