In 2023 the room used to ask whether retrieval-augmented generation was worth building. In 2026 the room assumes it is already there and asks which variant you picked. That is what a substrate looks like, the moment after it became one. The interesting work moved on; only the labels stayed.

I read this shift the way I used to read DJ booths in the late nineties. For a few years the only question was which turntable you owned. Then turntables became table stakes, and the only question that mattered was what was in your crate. RAG passed the same threshold. The retrieval pattern itself is no longer the differentiator. What documents you ground against, how you ground against them, and whether the user can verify the answer afterwards: that is the new conversation.

This piece sits inside the larger LLM harness, where retrieval is one of five sides. Where the harness asks “can we trust this system end to end”, this piece asks the narrower question: what does the grounding side look like, in 2026, with the variants the field has actually settled into.

The pattern, in one paragraph

Retrieval-augmented generation, first formalised by Lewis et al. in 2020 (“Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”), wraps a language model with a retrieval step. Instead of letting the model answer from whatever it memorised during training, the system first fetches relevant material from a corpus you control, then asks the model to answer using that material. The model becomes a reader and a writer, not an oracle. The corpus becomes the source of truth. The retrieval quality becomes the bottleneck. That last sentence is doing most of the work; almost every failure mode in modern RAG systems traces back to “the retrieval was bad and the model answered anyway.”

The family tree, where it diverged

The base pattern stayed intact. What changed is that the field grew variants for the situations the original 2020 paper did not anticipate. Three branches matter today.

Document retrieval optimises for the enterprise case: a corpus of documents (contracts, policies, manuals, tickets, knowledge base articles), chunked into passages, embedded into a vector database, retrieved by similarity. This is what most teams mean when they say “RAG” without qualifying it. The failure modes here are well understood now: poor chunking destroys recall, retrieval relevance does not equal answer relevance, and the context window quietly fills up with adjacent-but-irrelevant passages.

Parametric retrieval is the newer branch, embedding retrieval-style behaviour directly into model weights rather than the prompt context. The trade-off is speed and freshness on one side, transparency and updatability on the other. It is the active research front, not yet the default.

Graph retrieval treats the relationships between entities as first-class context, not just the documents that mention them. A pure vector retrieval over text loses the structure that says “this CEO worked at that company before founding this one.” A graph retrieval keeps it. Microsoft Research published one of the clearest framings of this in 2024 (Edge et al., “From Local to Global: A Graph RAG Approach”), and the pattern is gaining ground anywhere multi-hop reasoning over named entities is the actual job.

Citations sit underneath all three. They are not a fourth variant; they are the user-visible artefact that proves any of the three is on. Without a clickable trail from the answer back to the source, the grounding is invisible and the trust does not transfer. Citations are the moment retrieval graduates from an engineering choice into a product surface.

What the flow actually looks like in production

The retrieval-to-citation flow is the operational shape every serious RAG system converges on. The variants change what happens inside the “retrieve” and “ground” boxes; the shape of the flow stays the same.

Every box on that diagram is a place a hallucination can be born. Embedding the query wrong returns irrelevant material. Retrieving without reranking surfaces near-duplicates and pushes the actually-relevant passage out of the context window. Assembling grounded context without explicit citation slots means the model summarises without a return path. Generation without strict instruction to stay inside the retrieved passages lets the model interpolate from training data. Citation rendering that points to the wrong passage destroys trust the moment a curious user clicks. The discipline is wiring all seven boxes and instrumenting each one so the failure surfaces before the user sees it.

This is also the part of the harness where evaluation has matured fastest. Open-source frameworks like RAGAS formalised the metrics that matter on the retrieval side (faithfulness, answer relevance, context precision, context recall), and a serious team in 2026 runs them on production traffic, not just on a static test set. The deeper view of the evaluation discipline lives in evaluation and benchmarking; for the retrieval side, the relevant point is that the metrics now exist and the excuse for not measuring is gone.

The five pieces this section opens up

Each of these gets its own deep dive. The order below mirrors how I introduce the topic in client work: name the pattern, ground it in the dominant enterprise variant, look at the research front, surface the user-visible artefact, then add the structural extension that matters when relationships do the heavy lifting.

What RAG actually is, and why it became the substrate

The original Lewis paper proposed RAG as one technique among several for knowledge-intensive natural-language tasks. Six years later it is the default. The deep dive walks through what changed (cost economics, context-window expansion, vector database maturation, enterprise pressure to ground answers in proprietary data), why “RAG vs fine-tuning” turned out to be a false dichotomy, and where the pattern still fails (when the corpus itself is wrong, the answer is wrong with full confidence). If you are building today, this is the foundational piece. Continue reading on what RAG actually is, and why it became the substrate.

Document RAG for the enterprise: chunking, retrieval quality, the evaluation problem

The enterprise variant is where most production budget lands and where the failure modes are best documented. The deep dive covers chunking strategy (fixed-size, semantic, structural, late-chunking), retrieval quality measurement (faithfulness, context precision, answer relevance via frameworks like RAGAS), and the evaluation problem most teams discover on day forty-five (the retrieval looked great in development; in production half the queries return adjacent-but-wrong passages). This is the practitioner piece, with the failure modes named and the controls listed. Continue reading on document RAG for the enterprise.

Parametric RAG and the next grounding front

The active research front. Parametric retrieval folds retrieval-style behaviour into model weights themselves, trading the transparency of a separate retrieval step for speed and a different update profile. The deep dive frames what it is, what it is not (it is not “RAG with extra steps”; it is a different architecture), why it is rising in the research literature, and the questions a practitioner should ask before assuming it replaces document RAG (it does not, in 2026, for most enterprise use cases). Continue reading on parametric RAG and the next grounding front.

Citations as the user-visible proof of grounding

The harness component the user actually sees. The deep dive treats citations as a product surface, not an engineering afterthought: how to render them so the user trusts them, how to attribute them so they map back to passages and not just documents, how to handle the cases where the model paraphrases beyond what the citation supports (the citation-fidelity problem that destroys trust the moment a careful reader clicks). If retrieval is the engine, citations are the dashboard the driver reads. Continue reading on citations as the user-visible proof of grounding.

Graph RAG: knowledge-graph grounding as the next-gen retrieval substrate

The variant that treats relationships as first-class context. Most production RAG systems lose the structure between entities the moment they chunk a document into passages and throw them into a vector database. Graph retrieval keeps the structure and uses it for multi-hop reasoning, entity-context expansion, and the kind of question that requires three documents to be linked together to answer. The deep dive covers when graph retrieval is worth the maintenance cost (often less often than enthusiasts claim, but more often than document-only teams realise), where it intersects with the symbolic-grounding mitigations described in hallucination taxonomies and mitigations, and how my own knowledge graph fits as one worked example of the pattern. Continue reading on graph RAG.

What stays true across the family

Three things hold no matter which variant you pick.

The corpus is the source of truth. If the corpus is wrong, stale, or thin, every variant fails the same way: the model answers from a bad foundation with full grammatical confidence. The grounding does not fix the data; it only narrows the surface from which the wrong answer is generated. Data integrity (the upstream side of the harness covered in observability and audit) is the dependency every retrieval pattern inherits.

The retrieval step is the bottleneck. Once retrieval returns the right passages, modern models are very good at writing grounded answers from them. Once retrieval returns the wrong passages, no amount of model upgrading recovers. This is why the rise of evaluation frameworks for retrieval, separate from end-to-end evaluation of the generated answer, is the right shape of the field maturing.

Citations are the trust transfer. Without them, retrieval is internal-only and the user has to take the system’s word for it. With them, the user can verify, and the moment they verify the trust compounds. The product surface where retrieval meets user perception is the citation. Most teams under-build it because it does not look like AI work. It is.

Where this fits

Retrieval is one of the five sides of the LLM harness. On its own it is necessary, not sufficient: a perfectly grounded answer can still cross a guardrail boundary, still need the evaluation discipline to prove it works in production, still produce a systemic hallucination when the corpus contradicts itself, still need an audit trail for the regulator to reconstruct what was retrieved on which day. The point of the harness frame is that retrieval does not stand alone, and the point of this section is that retrieval, on its own terms, has matured enough to be reasoned about as a discipline with its own variants, failure modes, and evaluation metrics. The five deep dives below are the practitioner-level walkthrough.