Why use hybrid search instead of only vector search?

30-second answer

Dense embeddings capture meaning and paraphrase but are weak on exact tokens such as error codes, SKUs, names and rare jargon. BM25 excels at exact matches. Combining them (often with Reciprocal Rank Fusion) improves recall across both query types at little cost.

Try answering out loud before reading on. The walkthrough below is paced for a 40-minute interview round.

Architecture diagram

Sketch this on the whiteboard as you talk through the answer. See all AI architecture diagrams →

Scroll sideways to see the whole diagram.

Hybrid retrieval with rank fusion. BM25 and vector search run in parallel, Reciprocal Rank Fusion merges their rankings, and a cross-encoder picks the final few chunks.

0–5 min Clarify and frame

Framing: "Dense and sparse retrieval fail on different queries, so combining them improves recall cheaply. I'll explain each method's strengths, how to fuse them, and how to verify the gain."

5–15 min Core explanation

Dense (vector) retrieval embeds queries and documents into a semantic space.

Strengths: paraphrase, synonyms, natural-language questions, cross-lingual matching.
Weaknesses: exact identifiers (ERR_4012, SKU-88-21), rare names, new jargon the embedding model never saw, negation and precise numbers.

Sparse (lexical) retrieval like BM25 scores term overlap weighted by rarity (IDF) and length normalization.

Strengths: exact terms, codes, names; no training needed; interpretable.
Weaknesses: vocabulary mismatch ("car" vs. "automobile"), long conversational queries.

Real query logs contain both kinds, so either alone leaves recall on the table.

15–25 min Deep dive: fusion methods

Reciprocal Rank Fusion (RRF):

score(d) = Σ over retrievers  1 / (k + rank_r(d)),   k ≈ 60

Uses only ranks, so it doesn't need comparable scores. Robust and the common default.

Weighted score fusion: normalize each retriever's scores (min-max or z-score) then combine with weight α.

Can beat RRF when tuned, but scores shift with query type and corpus.

Learned sparse models (SPLADE-style) expand documents and queries with related terms, bridging the gap between the two.

Pipeline:

query → BM25 top 50 ┐
                    ├─ RRF → top 50 → cross-encoder rerank → top 5
query → ANN top 50  ┘

25–32 min Trade-offs

Cost: two indexes and two queries; usually small compared to the LLM call.
Complexity: keeping both indexes in sync on updates and deletes.
Store support: many databases and search engines now support hybrid queries natively, reducing complexity.
When dense alone is fine: short corpora of natural prose with conversational queries; measure before adding complexity.

32–37 min Production example

A developer-docs assistant failed on questions mentioning specific error codes and CLI flags: vector search returned conceptually related pages without the exact code. Adding BM25 with RRF fixed most of these. On their eval set, recall@10 improved notably for code-heavy queries without hurting conceptual ones, and the reranker handled ordering.

37–40 min Likely follow-ups

How do you weight the two? Start with RRF; if tuning, use an eval set segmented by query type.
Does hybrid replace a reranker? No; hybrid improves candidate recall, reranking improves top-k precision.
How do you handle multilingual? Multilingual embeddings for dense; language-specific analyzers for BM25.

What interviewers listen for

Concrete failure examples for each method.
RRF formula and why rank-based fusion is robust.
Evaluating with a segmented query set.

Red flags

"Vector search is always better than keyword search."