Retrieval-Augmented Generation (RAG), Simply Explained

Retrieval-Augmented Generation (RAG) is a technique utilized to decrease Large Language Model (LLM) hallucinations by allowing LLMs to access external or private data (by choice of the user) that was not included in training.

Say you ask ChatGPT (or the LLM of choice): “Should I drink milk as part of my breakfast?”

It should provide a detailed response as to milk’s nutritional benefits, and it’s likely that part of this response are scientific articles related to this exact subject. These huge models are able to access this information since they have “digested” millions of texts and other materials—ChatGPT 4 is said to have been trained on 13 trillion tokens, which is roughly 10 trillion words.

Their outputs are generic responses based on the model’s pattern recognition of what it sees in the vast sea of data that it has been trained upon.

But what if you ask ChatGPT or Claude: “What did the health newsletter in my email say about drinking milk in the morning?”

ChatGPT-4: ”I don’t have access to your personal email or its contents, so I can’t see what your health newsletter said about drinking milk in the morning.”