Advanced RAG: reranking, hybrid retrieval, and query rewriting

The leaderboard reflex

Chapter 5’s table is on the wall: the paraphrase forced a junk-or-abstain choice, the morphology trap beat both configs, and a chatty run-up chunk outranked the actual answer. In standup, the first proposal arrives within a minute: “Let’s switch to the top embedding model on the leaderboard.” It’s a real upgrade — re-embed 4.5M chunks, migrate the index, a week of work.

Predict, failure by failure: which of the three would the shiny new embedding model actually fix? And is there anything cheaper that your own golden set already points to?

Two-stage retrieval: cheap recall, expensive precision

First-stage retrieval — TF-IDF here, a bi-encoder in production — scores the query and each document independently: the document’s vector was computed before your query existed. That’s what makes it fast (embed once, search sub-linearly) and what caps its precision: it cannot read the pair. A cross-encoder reranker does exactly that — query and document go in together, attention reads them jointly — making it substantially better at judging relevance (typically worth +5–20 nDCG points over the first stage’s ordering) and far too slow to run over a corpus: it costs one forward pass per pair.

So production retrieval is a cascade: the cheap stage optimizes recall over 20–100 candidates; the expensive stage re-orders those candidates and keeps the top 5–10. Never rerank the corpus — at 50,000 documents that’s 50,000 forward passes per query, ten-plus seconds of latency for nothing the cascade wouldn’t give you. When even one reranker pass is too slow, cascade the cascade: a fast small reranker cuts 100 → 30, an accurate one cuts 30 → 10 — typically preserving ~98% of the quality at ~60% of the latency.

The companion’s reranker keeps the shape and swaps the learned scorer for a transparent one — what fraction of the query’s (folded, stopword-free) terms does the candidate actually cover?

def coverage_score(query: str, doc: str) -> float:
    q_terms = _informative(query)            # folded, stopwords dropped
    d_terms = set(fold_tokenize(doc))
    return len(q_terms & d_terms) / len(q_terms)

Same interface as Cohere Rerank or bge-reranker — score pairs, re-sort the candidates — so when you swap in the real thing, nothing else in the pipeline moves. Coverage is also exactly the signal the chatty-chunk failure needed: a document that covers 3 of 4 query terms now beats one that repeats “refund” five times, because the second stage reads the pair instead of trusting the corpus-wide statistics. Reranking is consistently reported as the highest-ROI retrieval upgrade — practitioner figures cluster around +15–25% recall@5 for 50–100ms, a range to verify on your own golden set rather than import — except when your latency budget is already spent or first-stage recall is already ~95%+; it re-orders candidates, it cannot resurrect what stage one never found.

Hybrid retrieval and rank fusion

Lexical and dense retrieval fail in complementary ways — that’s the entire case for running both:

• Lexical (TF-IDF/BM25) nails exact rare tokens — error codes, SKUs, acronyms, names — and misses paraphrase (“money back” vs “refund”).
• Dense (embeddings) nails paraphrase and concepts — and can blur the exact token that makes two SKUs different.

Hybrid search runs both and merges the rankings with Reciprocal Rank Fusion: each list votes $1/(k + \text{rank})$ for its documents, $k = 60$, votes sum. Fusing in rank space ducks an ugly problem — a BM25 score and a cosine score live on incomparable scales, and normalizing them is fragile — while rewarding documents that both retrievers liked:

One operational caveat: RRF degrades to single-retriever behavior when one list comes back empty (new vocabulary on the lexical side, say) — monitor per-retriever hit rates, not just the fused result.

Two more upgrades from this chapter’s kit are the same idea at smaller scale. Folding/stemming kills the morphology trap (“item” ≠ “items”, “exchange” ≠ “exchanged”) — and because Chapter 2 left the vectorizer as a seam, it’s one line: TfidfIndex(corpus, tokenizer=fold_tokenize). Our ten-line stemmer mangles plenty (production uses Porter/Snowball, or dense embeddings make it moot); the point is where the fix lives — the vectorizer, not the search machinery. And query expansion attacks paraphrase from the query side: search several phrasings, fuse with RRF. The companion’s variant map is hand-authored ("get my money back" → "request a refund"); production generates variants with an LLM — multi-query rewriting, or HyDE, which asks the model for a hypothetical answer and embeds that (answers resemble documents more than questions do). Both buy recall with an extra LLM call of latency before retrieval even starts — precision-critical, latency-tolerant systems only.

The routing zoo, priced

Past these, the named “advanced RAG” patterns are all one move — spend more compute deciding what to retrieve or whether to trust it — at escalating prices:

| Pattern | Mechanism | Price | Buy when | |---|---|---|---| | Self-RAG | model decides whether to retrieve, grades doc relevance, checks its own grounding | 2–3× LLM calls | many queries don’t need retrieval at all | | CRAG | grade retrieved docs; below threshold → rewrite query, re-retrieve, or fall back to web search | 1.5–2×, slow on fallback | retrieval quality is inconsistent and a fallback source exists | | Adaptive routing | classify query complexity → route to BM25 / hybrid / iterative retrieval | +1 small-model call | mixed simple/complex workload with tight latency | | Graph-RAG | entity extraction + knowledge-graph traversal alongside vectors | a graph DB to build and operate | multi-hop, relationship-shaped questions (“who reports to whom”) |

Read the table as a menu with prices, not a checklist. The buying discipline is the chapter’s actual lesson: let the golden set order the queue. Folding cost ten lines; expansion cost a variant map; a reranker costs ~80ms; Self-RAG costs every query 2–3× — and each purchase is verified by re-running the same exam, never by vibes. (The self-evaluating patterns — Self-RAG’s reflection, CRAG’s grading — are early forms of agentic behavior: a system inspecting its own intermediate results and choosing its next step. That generalizes far beyond retrieval, and it’s Chapter 8.)

See it: the same exam, after the upgrades

Config B is Chapter 5’s hardened baseline. Config C adds the kit you just built — folded vectorizer, query expansion, RRF fusion, coverage reranking, a floor measured in coverage units — and reuses the same coverage scorer at answer-extraction time, so the upgrade reaches the last ranking decision too. Predict before revealing: which of the four red cells in B’s column move to green — and which failure survives every retrieval upgrade, and why?

How this is graded

• Technical Correctness — you can say why a cross-encoder out-judges a bi-encoder (it reads the pair), why RRF fuses ranks not scores, and where stemming lives (the vectorizer).
• Trade-off Awareness — this chapter’s weighted dimension: every upgrade quoted with its price (latency multiplier, extra calls, operational burden) and matched to a measured failure, cheapest-first.
• Evaluation Rigor — each purchase re-verified against the same golden set; “we added a reranker” means nothing without the before/after table.
• Communication — “folding and a coverage reranker fixed the morphology and ranking failures for ~0ms; HyDE stays on the shelf until a measured paraphrase gap survives hybrid search” beats reciting the pattern zoo.

Industry variation

• E-commerce & enterprise search — hybrid is table stakes: SKUs, part numbers, and acronyms make pure-dense retrieval naive at any serious level; pgvector + tsvector RRF recipes ship this in a sprint.
• Latency-SLA products — cascades and selective reranking are the norm; rerank only low-confidence queries, batch aggressively, and track per-stage p95 like a budget line.
• Compliance-bound domains — CRAG’s web-search fallback is often forbidden (the answer must come from the governed corpus); the corrective action becomes query rewriting and an honest abstain, and Graph-RAG earns its ops cost where relationship queries dominate (ownership chains, reporting lines).

Stretch: the bill arrives

Config C ships. Quality holds. Then the invoice and the latency dashboard land in the same week: expansion doubled retrieval calls, the reranker added its milliseconds, the retrieval stage went from 180ms to 520ms — and end-to-end p95 blew past the 2-second SLA while cost per query tripled. Nobody can say which stage spends what. Before anything gets cut, what would you instrument — per stage — and which three numbers decide what lives? That’s Chapter 7: RAG in production, where latency budgets, token economics, and caching turn the pipeline you built into something a business can run.