Home / Question bank / RAG & Vector Search

RAG & Vector SearchMidQuestion 12 of 12

What is HyDE?

30-second answer

Hypothetical Document Embeddings: have the LLM write a plausible answer to the query, embed that, and search with it. A hypothetical answer often lies closer to real answer passages in embedding space than a short question does. It costs an extra LLM call and can drift if the hypothetical answer is wrong.

Try answering out loud before reading on. The walkthrough below is paced for a 40-minute interview round.

0–5 min Clarify and frame

Framing: "HyDE addresses the mismatch between how questions and answers look in embedding space. I'll explain the idea, when it helps, its costs and risks, and related query-transformation techniques."

5–15 min Core explanation

Short queries ("how to rotate keys?") and long documents ("To rotate your API keys, open Settings → Security…") are phrased differently, so their embeddings may not be close.

HyDE (Hypothetical Document Embeddings):

  1. Ask an LLM to write a plausible answer passage for the query, without retrieval.
  2. Embed that hypothetical passage.
  3. Use its embedding to search the corpus.
  4. Generate the final answer from the real retrieved documents.

The hypothetical answer may be factually wrong, but it uses answer-like vocabulary and structure, which tends to land near real answer passages.

Technique Idea
Query rewriting Turn a follow-up into a standalone question using chat history
Multi-query Generate several paraphrases; retrieve for each; fuse results
Query decomposition Split multi-part questions into sub-questions
Step-back prompting Ask a more general question first to retrieve background
HyDE Embed a hypothetical answer instead of the question

These can be combined: rewrite, then decompose, then retrieve with HyDE per sub-question, then fuse.

25–32 min Trade-offs

  • Latency and cost: an extra LLM generation before retrieval (hundreds of ms or more).
  • Drift: if the model's guess is badly wrong or off-topic, retrieval follows it. Mitigate by combining HyDE results with standard query results.
  • Less useful for exact-term lookups (IDs, codes); BM25 handles those better.
  • Domain knowledge: for very specialized domains the model can't guess the shape of answers well.

32–37 min Production example

A research-paper search tool had poor recall for short natural-language questions. Adding HyDE plus the original query with RRF improved recall@20 on their eval set. Since it added ~400 ms, they only triggered HyDE for short queries without exact technical terms, keeping latency low for the rest.

37–40 min Likely follow-ups

  • Could you precompute instead? Yes: generate hypothetical questions for each chunk at index time and embed those, matching question-to-question at query time.
  • Does HyDE help with BM25? Expanding the query with generated terms can help lexical search too.

What interviewers listen for

  • The question–answer asymmetry explanation.
  • Cost, latency and drift trade-offs.
  • Knowing alternatives and combining with original queries.

Red flags

  • Using the hypothetical answer itself as the final answer.