Each pattern, full-width, with its architecture sketch, the stack we'd deploy, the accuracy work that gates it, and an honest read on when it earns the build. Five are the patterns we reach for in most production conversations. The rest earn the build in narrower shapes of corpus or query.
Primary patterns are colour-coded by brand gradient. Specialised patterns sit on neutral cards. Every card links to the full playbook: deployment, stack, accuracy work, community verdict, and our take.
Embed the query, fetch top-K, stuff them in the prompt. No rewriting, no fusion, no reranking.
Single-domain FAQ, internal docs search, narrow-scope chatbots where queries are short and the corpus is well-keyed. The fastest thing that can possibly work.
Anything with ambiguity, jargon mismatch, multi-hop reasoning, or a corpus larger than ~50k chunks. Recall falls off and the LLM hallucinates the rest.
Naive RAG plus a conversation buffer. The model can resolve pronouns and follow-up references because it sees prior turns.
Customer support chat, tutoring bots, personal assistants where the second question depends on the first. Cheaper than rebuilding context every turn.
When privacy or retention requirements rule out persistent state, or when the conversation has no useful continuity (one-shot search).
Compose the pipeline from swappable parts (retriever, reranker, generator). Not a pattern itself; an engineering posture.
Every production build. Module boundaries let you swap a reranker or change the embedding model without rewriting the whole thing. The 2026 default engineering posture.
Prototypes where shipping in two days matters more than swapping a component in two months.
Query rewriting + hybrid (vector + BM25) + reciprocal-rank fusion + cross-encoder rerank + citation discipline. The 2026 production consensus.
Most production RAG. Reranking alone lifts Recall@5 by ~17 points; hybrid catches the exact-term matches embeddings miss. Worth the extra ~30ms p95 on almost every build.
Throughput-extreme workloads where you cannot pay the rerank latency tax, or corpora so well-keyed that hybrid + rerank doesn't move the needle.
LLM decides what to retrieve, from which source, and whether the result is good enough. Multiple retrieval rounds, source-specific agents, validation step.
Heterogeneous corpora, multi-source research (legal + financial + internal), complex queries that need decomposition. Where one shot of retrieval was always wrong.
Cost-sensitive volume traffic. Each query costs multiple LLM calls. Latency is unpredictable.
Build a knowledge graph over entities + relationships. Retrieval becomes graph traversal. Reasoning becomes path-following.
Multi-hop reasoning across linked facts: clinical decision support, regulatory analysis, complex case files, investigative journalism. Published numbers report 81%+ accuracy in specialised domains, +6.8 pts over flat RAG.
Flat document collections, fast-changing data, small corpora where the graph build cost dwarfs the retrieval gain.
Model evaluates its own retrieval and answer. Decides if it needs to retrieve again, with what query, before responding.
Vague or under-specified queries, domains where a confidently wrong answer is more expensive than a slow one. Hides bad retrieval from the user.
Latency-critical interactive use. Self-critique adds round-trips. Can also refuse to answer too often when uncertainty is high.
Run multiple interpretations of the query in parallel, score each, pick or merge the best answer.
Open-ended research queries, comparative analysis (this product vs that), domains where the question has multiple legitimate framings.
Cost-sensitive workloads, since branching multiplies LLM + retrieval calls. Can overwhelm the user if results aren't filtered.
Classify the query (simple / complex / broad / narrow) and route to a matched retrieval strategy. Simple queries get fast pipelines; complex ones get the agentic shape.
Mixed-shape traffic: public-facing assistants, support bots, internal tools that see everything from "what's our return policy?" to "compare these three contracts".
Single-shape workloads where every query is the same depth. Adds classifier latency without earning it back.
Predict the next likely query while answering the current one. Pre-fetch retrieval for the predicted follow-up.
Latency-critical interactive use where the conversation has predictable shape. Autocomplete-style search, support flows with well-known follow-ups.
Open-domain or one-shot use. Wrong predictions waste compute and (worse) prime the cache with irrelevant context.
After retrieval, evaluate document quality. If weak, fall back to web search or query rewrite before generation.
High-stakes accuracy contexts like legal research, academic writing, and policy analysis, where catching a bad retrieval before generation is worth the extra latency.
Volume traffic. Quality checks add cost on every query, including the ones that would have been fine.
LLM writes a hypothetical answer to the query, embeds the answer, retrieves documents semantically similar to it. Semantic matching, not term matching.
Technical or specialist domains where the query and the document use different vocabulary. Medical, legal, academic. When BM25 misses and embeddings need the right anchor.
Domains where the LLM is likely to hallucinate the hypothetical. The bad guess pulls retrieval down a wrong path. Always pair with rerank.
Text + images + tables + audio in one retrieval surface. Vision LLMs and multi-modal embedding models (BGE-M3, ColPali) make documents understandable, not just searchable.
PDFs with tables and figures (legal, financial, technical), visual catalogs, scanned archives, medical imaging notes. See our /llm/rag/multimodal/ playbook for the full build.
Plain-text corpora. The extra extraction step costs latency and complexity without retrieval gain.
Generate, retrieve based on the partial answer, generate again. Used inside agentic and chain-of-thought workflows when one retrieval pass isn't enough.
Long-form synthesis, structured report writing, queries that decompose into sub-questions. Often a composition pattern inside agentic systems rather than a standalone build.
Single-turn lookup. Iteration just adds latency.
pgvector, Qdrant, Milvus, Weaviate, Vespa, LanceDB, Pinecone. Honest 2026 comparison and our default.
Read moreEmbeddings, chunking, hybrid search, reranking. The four layers retrieval quality lives or dies in.
Read moreProven designs from under 100k chunks to over 1B. The architecture changes with the scale.
Read morePDFs with tables and figures. Vision LLM extraction, ColPali, BGE-M3, court-ready citations.
Read more