Hu Jin

Posted on Nov 12 Edited on Nov 12

Hybrid Search in RAG: Combining BM25 Keyword and Vector Retrieval

#ai #rag #search #embeddings

Vector search finds meaning. Keyword search finds exact words. Most production RAG systems need both. A product code like SKU-4412, a person named Nguyen, or an error string from a log will often lose to a paraphrased but irrelevant chunk if you only compare embeddings. Hybrid search runs both paths and merges the results so the model sees passages that match on terms and on intent.

Why keyword search still matters

Dense embeddings are great at synonyms and paraphrases: "cancel my plan" can match a chunk about "subscription termination." They are weaker when the query hinges on a rare token that must appear literally.

BM25 (and similar sparse scorers) reward term frequency and rarity. If the user types an ID, a legal clause number, or a function name, BM25 puts those documents near the top even when the surrounding wording differs from the query.

The two retrieval paths

Keyword path. Tokenize the query, score documents with BM25 (or Lucene/Azure Search text scoring), and keep the top n. Strengths: exact matches, acronyms, codes. Weaknesses: vocabulary mismatch and synonyms.

Vector path. Embed the query with the same model you used for chunks, then run nearest-neighbor search (HNSW, IVF, and so on). Strengths: semantics and paraphrase. Weaknesses: rare tokens and short identifier queries.

Running both is cheap compared with generation. Fetch a larger candidate set from each path (for example, 20–50), then fuse.

Fusing the ranked lists

You rarely want raw score arithmetic: BM25 scores and cosine similarities live on different scales. Rank-based fusion is the usual fix.

Reciprocal Rank Fusion (RRF) is a common default. For each document d across result lists:

score(d) = Σ 1 / (k + rank_i(d))

where rank_i(d) is the 1-based rank of d in list i, and k is a small constant (often 60). Documents that appear high in either list rise; documents that appear in both rise further.

Alternatives include weighted sum after min-max or z-score normalization, or learning a small ranker later. Start with RRF; measure; then tune weights if one path consistently dominates your test set.

When to lean which way

IDs, SKUs, error codes, proper names: keyword-heavy or hybrid with a higher keyword weight.
Conceptual questions ("how do we handle refunds for annual plans?"): vector-heavy hybrid.
Mixed enterprise queries: hybrid as the default, then a cross-encoder reranker on the fused top 50.

Many managed search products (Azure AI Search, Elasticsearch, OpenSearch) expose hybrid queries in one call. Use that when you can so scoring, filters, and security trimming stay consistent.

Practical tips

Same filters on both paths. Apply ACL, tenant, and date filters before fusion so you never merge a forbidden hit.
Same index, two fields. Store content as searchable text and vector as the embedding; query both.
Don't over-retrieve into the prompt. Hybrid improves recall; still pass only the best few chunks to the LLM (or rerank first).
Evaluate separately. Track recall@k for keyword-only, vector-only, and hybrid on a labeled set. Hybrid should win overall; if it doesn't, check chunking or embedding mismatch.

Minimal mental model in code

# Pseudocode — Azure AI Search style hybrid
results = search.search(
    search_text=question,  # BM25 / keyword
    vector_queries=[VectorizedQuery(vector=qvec, k_nearest_neighbors=50, fields="vector")],
    top=10,  # fused top results after RRF-style merge inside the service
)

With separate stores you implement RRF yourself: take the union of IDs, sum reciprocal ranks, sort, cut to k.

Hybrid search is the boring upgrade that quietly fixes a large class of RAG failures. Keep embeddings for meaning, keep BM25 for exactness, fuse by rank, and measure on queries that look like your users—not only on paraphrase benchmarks.

About the author

I write about Applied AI, MCP, Azure AI Search, and RAG. Follow me on Dev.to and Hashnode for more.

DEV Community