Hybrid search for RAG: keywords plus embeddings

Hybrid search runs two retrievers over your documents at once: a keyword retriever that scores exact word matches, and an embeddingA list of numbers that captures the meaning of a passage, so two pieces of text about the same idea land close together in that number space. retriever that scores meaning. You then merge the two ranked lists into one. The keyword side catches literal strings like error codes and product IDs; the embedding side catches paraphrase and concept. Together they cover the cases each one misses alone.

If you have not read RAG fundamentals yet, start there. This guide assumes you already chunk, embed, and retrieve, and want the retrieval stage to stop dropping exact terms.

Why is one retriever not enough?

Pure embedding search has a known blind spot. It scores passages by how close their meaning is to the question, so it is strong on paraphrase and weak on rare literal strings. Ask for ticket INC-4821 or the flag --no-verify and the embedding for that token sits near every other ID or flag, so the right passage does not rise to the top.

Pure keyword search has the opposite blind spot. BM25A keyword-ranking algorithm that scores a document by how many query words it contains and how rare those words are across the collection. Strong on exact terms, blind to meaning., the standard keyword ranker, scores a document by the query words it contains, weighted by how rare each word is. It nails exact terms and is blind to synonyms: search "car" and a passage that only says "vehicle" scores zero.Common gotchaThe exact-term win only holds if your analyzer keeps IDs whole. A default tokenizer splits INC-4821 and parseInvoice into pieces, so the literal match you rely on never happens.Index identifier fields with a keyword or whitespace analyzer, not the standard text one.

Hybrid search takes the union: run both retrievers, then combine. You keep BM25's exact-term precision and the embedding model's reach over meaning.

How do you combine BM25 and embedding scores?

You cannot add a BM25 score to a cosine similarity score. They live on different scales: BM25 is an unbounded positive number that depends on word rarity, while cosine similarity sits between -1 and 1. Summing them lets one retriever drown out the other for reasons that have nothing to do with relevance.Common gotchaThe trap is adding BM25 and cosine scores straight into one number because both "feel" like relevance. A rare-word BM25 hit can score 18 while every cosine value is under 1, so the keyword side silently decides every ranking and the embedding half stops mattering.Either min-max normalise both lists into 0 to 1 before any weighted sum, or skip raw scores entirely and fuse on rank with RRF.

Two approaches solve this.

Score normalisation

Rescale each retriever's scores into the same range, often 0 to 1, then take a weighted sum. A weight near 0.5 treats both retrievers equally; push it towards the keyword side for a corpus full of IDs and codes, towards the embedding side for prose. Normalisation works, but it is sensitive: the rescaling depends on the score distribution in each batch of results, so the same document can land at a different fused score depending on what else came back.

Reciprocal rank fusion

Reciprocal rank fusionA fusion method that combines ranked lists using only each document's position, not its raw score, so retrievers on different scales merge cleanly. (RRF) sidesteps the scaling problem by ignoring raw scores. It looks only at the position a document holds in each list. The fused score for a document is the sum, across both lists, of 1 / (k + rank), where rank is the document's position (starting at 1) and k is a constant.

k is set to 60 in most implementations, a value that has held up well across datasets. It dampens the gap between the top ranks so that a document ranked second is not buried under one ranked first, while a document that appears high in both lists still wins overall. Because RRF reads positions and not scores, you do not need to know either retriever's score distribution, which is why search engines reach for it as the default hybrid fuser.

Worked example. A document at rank 1 in keyword search and rank 3 in embedding search scores 1/(60+1) + 1/(60+3), about 0.0164 + 0.0159 = 0.0323. A document that is rank 1 in embeddings but absent from the keyword list scores only 1/(60+1), about 0.0164. The document that shows up well in both lists ranks higher than the one that wins a single list. That is the behaviour you want: agreement across two independent signals is a stronger relevance vote than a single high score.

Should you add a reranker after fusion?

Fusion gives you one ranked list. A rerankerA second-pass model that reads the question and each candidate passage together and re-scores them, so the most relevant passage moves to the top. then reads the question and each top candidate together and re-scores them. This is a different operation from the first-pass retrievers, which score the question and documents separately. Reading them as a pair catches relevance that neither BM25 nor a single embedding can see, at the cost of one extra model pass over a short list.

The pattern that holds up: retrieve a wide net with hybrid search (say the top 50 fused results), then rerank down to the handful you send the model. Wide recall first, sharp precision second.

Build it with Claude Code

Hybrid search is plumbing: two queries, a fusion function, a final rank. Claude Code writes that code, and your job is to specify the behaviour precisely so the result is correct rather than plausible. Be concrete about which fields are indexed, which fuser you want, and what "done" looks like.

Asking for the test matters. Hybrid retrieval has a precise, checkable property: a document that ranks well in both lists must beat one that ranks well in a single list. A test pins that down so a later change cannot quietly break it.

When you want to inspect what each retriever returned before fusion, ask Claude Code to log both lists side by side. Seeing that BM25 found the exact ID while the embedding search found the paraphrase is how you confirm both halves are pulling their weight rather than one silently returning nothing.

Plan first on a real corpus

For anything beyond a toy index, decide the shape before you write code. Ask Claude Code to plan, not build, on the first pass.

A plan surfaces the decisions that move quality: field choice, candidate counts, and whether the reranker earns its latency on your corpus. Settle those, then have Claude Code implement against the plan.

Connect a search backend over MCP

If your keyword and vector indexes already live in a search engine or database, expose that engine to Claude Code through an MCP server so it can query the real index while it builds and tests, instead of guessing at your schema.

Add a remote MCP server from the terminal:

claude mcp add --transport http search-engine https://mcp.example.com/mcp

Or declare it in a project .mcp.json so everyone on the team gets the same connection:

{
  "mcpServers": {
    "search-engine": {
      "type": "http",
      "url": "https://mcp.example.com/mcp"
    }
  }
}

With the index reachable, Claude Code can run a query both ways, show you the two ranked lists, and fuse them, all against your actual data. See install your first MCP server for the full setup walkthrough. If a server fails to attach, fix an MCP server that won't connect covers the usual causes.

How do I know hybrid search is working?

Evaluate retrieval on its own, before you judge the answer. Build a small set of questions where you know the correct passage, including a few that hinge on an exact string and a few that hinge on a paraphrase. Then measure:

• Recall at k. Of the questions whose correct passage exists, how often does it land in the top k? This is the number that matters most. If the right passage is not retrieved, the model cannot use it.
• Exact-term coverage. On the ID and error-code questions, does the fused list include the literal match? If not, your BM25 half is misconfigured or its weight is too low.
• Paraphrase coverage. On the synonym questions, does the embedding half surface the right passage even when the query words never appear in it?

Run that set before and after each change. Hybrid search is dependable once both halves are pulling and the fusion is rank based, and a fixed evaluation set is how you keep it that way.

Next, read RAG for coding agents, where exact identifiers dominate and the keyword half of hybrid search does most of the work.