Summary
Retrieval-Augmented Generation combines a language model with an external knowledge source. The model retrieves relevant documents at query time and generates its response conditioned on them. RAG is the standard pattern for any application where the answer depends on knowledge outside the model's training set.
Objectives
- 01State when RAG is preferable to fine-tuning.
- 02Sketch the minimal RAG pipeline.
- 03List the failure modes of naive RAG.
The Lesson
Why not fine-tune
Fine-tuning bakes knowledge into weights — expensive to update, hard to attribute, prone to hallucination on adjacent facts. RAG keeps knowledge in a database — easy to update, easy to cite, and the model only needs to synthesize what it just read.
Key Ideas
- RAG for knowledge that changes; fine-tuning for behavior.
- The knowledge lives in the retriever. The synthesis lives in the model.