# The Math-Based Guide to RAG, MCP, and Agents #108 Jun 29 Agents https://www.aibutsimple.com/p/the-math-based-guide-to-rag-mcp-and-agents After pre-training, a large language model is considered frozen, with its knowledge sealed inside its weights. At this point, the transformer base itself is only able to take inputs and sample output tokens, meaning it can’t search the web, execute multi-step goals by itself, and access a specific company's documents or data. These are notable limitations if you want a model to do real work. Suppose an employee asks an LLM: "Is this customer eligible for a refund?" A model can't answer this on its own. - It doesn’t have the company refund policy, - It can't look up the order or interact with the company systems, - And it can't reliably carry out a multi-step workflow from start to finish. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/9d23005c-4c61-4165-9b21-d093b5a3a85b/RAG__MCP_and_Agents__9_.png) We present 3 techniques that remove a constraint from foundational LLMs: - Retrieval-Augmented Generation (RAG) gives it knowledge it doesn't have to access information in an efficient way. - Model Context Protocol (MCP) gives it tools it can't reach, to act on systems. - Agents give it a loop to pursue a goal across many steps. They build on each other and combine into a system that can actually tackle real, complex problems. --- ### What You’ll Learn 1. Why a trained LLM is fundamentally limited 2. RAG: retrieving knowledge by meaning and the math of similarity search 3. MCP: the protocol that standardizes how a model uses tools 4. Agents: wrapping the model in a goal-seeking loop 5. When to use each technique 6. How the three compose into a single system ### What’s Helpful to Know - Embedding - A vector that represents a piece of text's meaning. - Context window - The fixed budget of tokens a model can see at once. - Tool or function call - A model outputting a structured request (a name plus arguments) that an external system runs. - JSON-RPC - A standard format for sending a function call and obtaining a result. - Policy - A policy is a function that maps states to actions (a distribution over actions). ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/110b2b73-430d-4acd-be8b-e654cc90516c/RLVR_Overview__2_.png) - [Reinforcement Learning (RL)](https://newsletter.aibutsimple.com/p/reinforcement-learning-for-large-language-models-simply-explained) - A field of machine learning that focuses on training models to act in environments by maximizing a reward signal. --- ### Retrieval Augmented Generation (RAG) The term “RAG” is used everywhere in AI, gaining widespread popularity and adoption for its utility and simplicity. But what actually is it? As we mentioned previously, the LLM base itself is unable to see company documents or orders in a system, since that knowledge lives outside its weights. Retrieval-Augmented Generation (RAG) solves this problem by fetching relevant context to feed into a prompt. The “augmented generation” part of RAG simply means the model provides grounded answers since it can read the added context along with the prompt. From here, the real problem becomes how to find the right context to fetch. Why use RAG rather than giving the model all context with a prompt? You might have done simple context injection before, where you ask an LLM a question and attach a document to give it the context needed to answer that question. But given a limited context window, we can’t simply give an LLM all company documents and data with every prompt. This is where the “retrieval” part of RAG comes in. If we can retrieve only the context relevant to the prompt, we no longer need to worry about a limited context window. --- ### Why Keyword Search Isn't Enough The obvious approach to retrieving the relevant context is keyword matching: the user asks "How do I reset my password?" so we look for documents that mention “password” or “reset”. The problem with keyword search is that the relevant company document might not include the exact words in the search query, for instance, a document containing "recovering your login credentials." This page has the same intent but zero shared words with the prompt, so a keyword search won’t be able to find that relevant document. Thus, we need to search by meaning, not by exact words. To do that, we have to turn meaning into something we can measure. --- ### Embedding Models This semantic meaning feature is exactly what embeddings do. An embedding model splits documents into chunks that are then converted into vectors in a vector space, placed so that vectors with similar meanings land close together. We then use a vector database to store, organize, and search those vectors at scale. "Reset password" and "recover login credentials" end up near each other, even with no words in common. Now, finding text that is similar just involves finding vectors that are close in distance. To rank how similar two pieces of text are, we score the angle between their vectors using cosine similarity: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/317b6fb4-b7bf-47d7-abd6-005ffcfacc24/image.png) Here, sim(q, d) represents the similarity score, q is the query vector (prompt embedded as a vector), d is a stored document (or chunk) vector, q · d is their dot product, and ‖q‖ and ‖d‖ are their magnitudes. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/ec39109c-d728-40f0-ba6e-3822780ce2b5/RAG__MCP_and_Agents.png) This score is normalized by the vector lengths, so the size of the text doesn’t influence its similarity to the query. The score runs from 1 (same direction) through 0 (orthogonal) down to -1 (opposite direction). --- ### Finding The Best Matches Fast We have to score the prompt vector against every stored vector and keep the top k, the k stored vectors with the highest sim(q, d). The chunks corresponding to the vectors then become the context we hand to the model along with the original prompt. However, there’s a problem: scoring the prompt chunk against every stored chunk is slow, considering vector databases can have tens of millions of embeddings. This is where vector databases use Approximate Nearest Neighbor (ANN) search. Algorithms like Hierarchical Navigable Small World (HNSW) find the closest vectors without checking them all. Exact search costs roughly O(N) per query, but ANN empirically scales closer to logarithmic by exploiting the structure in how the vectors are connected. You get matches much faster, and the accuracy lost is minimal. --- ### The Full RAG Pipeline Putting it together, RAG has two phases: - Offline (done once, ahead of time): Documents are chunked and embedded as vectors. - Online (every query): Prompt/query is embedded, top-k similar chunks are retrieved. The relevant context is passed to the model for grounded answers. In practice, combining both vector search and a keyword search algorithm such as [BM25](https://en.wikipedia.org/wiki/Okapi_BM25?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents) using [Reciprocal Rank Fusion (RRF)](https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents) is the industry standard retrieval method. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/fef37147-f39c-491f-a5e9-6a79dda47085/RAG__MCP_and_Agents__8_.png) This is known as Hybrid Search, and it combines the best of both methods. - Vector search to retrieve content based on conceptual similarity. - Keyword search to catch exact codes, SKUs, and terms specified in a prompt that could otherwise be missed by embeddings. Vector and keyword search both return their own ranked lists. RRF scores a chunk by its rank in each list by computing 1 divided by the rank, plus a constant k (typically k=60), adding those scores across both lists. Chunks that rank well in both end up with the highest combined score. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/d2828935-42a2-4327-9978-eca83773d85f/image.png) In the formula, ri(d) is the rank of chunk d in list i, and the sum runs over both lists. It uses rank position rather than the raw similarity, allowing the two methods to combine on the same scale. For our refund question, this is the first step: RAG retrieves the refund policy the model was never trained on. But the policy alone doesn't tell us whether this customer's specific order qualifies. For that, the model needs to reach into a live system. That's what MCP is for. --- If you enjoy our content and custom visuals, consider sharing this newsletter with others or [upgrading](http://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents) so we can keep doing what we do. ### [Code AI Projects With AI, But Simple PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents) ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/963eba00-ab31-4912-8c1f-1470a52f9ecc/aibutsimple_academy__2_.png) Get elevated access with the PRO newsletter tier: - Exclusive Hands-on Tutorials With Code - Real-World Interview Questions and Cases (From Real Companies) - Longer Articles With Full, Step-by-Step Explanations and Beautiful Visualizations - Ad-Free Reading for an Uninterrupted Learning Experience [Stay Ahead in AI With PRO](https://www.aibutsimple.com/pro?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents) --- ### Model Context Protocol (MCP) RAG let the model read context it didn't have. But that context isn't enough and not always necessary. For a refund, the model doesn’t only need to read the company’s refund policy document; it also needs to query an orders database and pull the specific record. However, the model is capable of only emitting text: it cannot open a database, hit an API endpoint, or execute a script. MCP allows the model to interact with external tools, data sources, and applications in a standardized way. The model is able to output a structured request with a tool name plus arguments. Then, an application around the model can detect that request, run the function call, and give the result back to the model. Tools that the model can use can range from a database, an API, a calendar, a script, or even the RAG pipeline. However, tooling turns out to be surprisingly hard to do well. --- ### The Integration Problem Before any standard, every convention was built separately. If you wanted three different models to each use four different tools, you wrote a custom connector (the thing that recognizes the request, runs the real function, and formats the result) for every pairing. That's N models x M tools = N x M separate integrations. Add one new tool and you have to wire it into every model. Add one new model and you have to wire in every tool. The cost grows as a product, and it doesn't scale. The Model Context Protocol (MCP) defined one standard interface so that everything that speaks it can connect. Build a tool once as an MCP server, and any model’s client that speaks MCP can use it. Now each model implements the protocol once, and each tool implements it once. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0ba59c4c-e311-4bc8-8850-704b48c09651/RAG__MCP_and_Agents__7_.png) The math flips from N x M to N + M, so integrations grow by addition instead of multiplication. This is the problem MCP was designed to solve. --- ### How MCP Is Structured MCP uses a client-server architecture with three roles: - The host is the application the model runs in (a chat app, an IDE, an agent). - Inside the host, clients manage connections, with one client per server. - Each server wraps a specific tool or data source and exposes what it can do. A server exposes three kinds of things, called primitives: - Tools: executable functions the model can invoke (look up an order, send an email). - Resources: read-only data the model can pull in (a file, a database record). - Prompts: reusable templates for common interactions. When the model needs to act, it makes a structured request in JSON-RPC (a standardized lightweight format) from the server, which executes the request and returns structured results to the model’s context. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/dfc86a2d-c9b0-4ead-afed-59ff32fcb595/RAG__MCP_and_Agents__6_.png) With MCP, the model's reach extends to any system that has a server, integration cost grows linearly instead of as a product, and a tool built once works with every model that speaks the protocol. For the refund question, this is the second piece. MCP is how the model queries the orders database to pull the customer's actual record. We still have a final limitation, though. The model reads the policy, looks up the order, and makes a decision. Something has to drive the loop, deciding what to do at each step. That's the agent. --- ### Agents Each MCP tool call is still just a single exchange, RAG being one of those tool calls. Take our refund question: "Is this customer eligible for a refund?" Answering it means reading the policy, looking up the order, then checking it against the policy, consisting of multiple steps, each step depending on what the last returned. A single forward pass can't do this. The model needs to act, see the result, and decide again. An agent wraps the model in a loop. The model reasons about what to do next, takes an action, observes the result, and repeats until the task is done. This pattern is called ReAct (Reason + Act). ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/00411d36-6e47-4906-9cb4-f954962a79a8/Agentic_Loop.png) Crucially, this is where all three ideas unify. The "Act" step is usually an MCP tool call. The agent acts as the loop, MCP is how it acts, and RAG is one thing it can reach for. --- ### Modeling The Loop To talk about this precisely, we’ll follow the standard Markov Decision Process (MDP), the formalism for sequential decision-making. An MDP has four pieces: - a state s: everything the agent knows right now - an action a: what it does next - a transition that produces the next state from the current one and the action taken - a reward R: how good the outcome was ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/87dcf1e9-63d1-4a0d-8a73-7a0bea7e1ca2/MDP_in_RL.png) The agent's behavior is a policy π(a | s): given a state, it picks the next action. The goal of an MDP is to choose actions that maximize the expected reward over the whole task: ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/cf84dc49-b80b-4f5e-a5f6-c367462dbd45/image.png) Here, st and at are the state and action at step t, T is the final step, and γ (between 0 and 1) is the discount factor that weighs near-term vs. far-off reward. The expectation E[·] averages over the randomness in the agent's own choices and in what its tools return. The equation essentially says a good agent picks actions that add up to the most reward by the end, not just the best single step. Recall the LLM is still a frozen model, so it is not being trained on this objective while it runs. It is being prompted to behave like a goal-seeking policy. The MDP is the framing the agent approximates, not an objective it actually optimizes at inference. --- ### Connecting back to RL The same objective J(π) is what reinforcement learning optimizes, including the RLHF we covered in the [RL for LLMs](https://newsletter.aibutsimple.com/p/reinforcement-learning-for-large-language-models-simply-explained) issue, where a model was trained to maximize a reward signal. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/528165cf-d46d-4948-8427-9ae358fbd20f/LLM_RLHF_Pipeline__1920_x_1200_px___3_.png) The shared idea is policy optimization: choosing actions to maximize reward. The difference is that RLHF does it to the weights during training, while a ReAct agent leans on a frozen model to approximate the same behavior at inference. Wrapping a model in a loop turns single predictions into multi-step problem-solving. The agent can recover from a bad intermediate result, chain several tools together, and pursue goals it could never reach in one forward pass. That's the whole stack: an agent that loops, acting through MCP, reaching for RAG when it needs to read. --- ### How RAG, MCP, and Agents Fit Together We've now built three pieces. The satisfying part is that they aren't competing techniques; they're three layers of one system. For a refund: 1. The agent receives the goal and reasons about the first step: it needs the refund policy. 2. It acts through MCP, calling a retrieval tool, which runs RAG, embedding the question and pulling the relevant policy chunks. 3. It observes the policy, then reasons again: now it needs the specific order. It acts through MCP again, this time querying the orders database. 4. It observes the order, reasons one last time, and checks it against the policy. 5. The goal is met, so the loop stops and the agent answers. So the stack is clean: the agent is the loop that decides and persists, MCP is the standardized way it acts, and RAG is one of the tools it can reach for. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/13bd80d3-f2c4-43dd-b9a7-64abbd100320/RAG__MCP_and_Agents__10_.png) Mathematically (for our case), the agent is the policy, MCP provides an interface for the action space, and RAG is one action that pulls new knowledge into the state. --- ### What to Take Away A frozen model has three limits, and each technique removes one: - RAG fixes knowledge. Retrieve text by meaning using cosine similarity, then ground the answer in what was retrieved, cutting hallucination and keeping knowledge fresh without retraining. - MCP fixes tools. One standard protocol turns the N x M tangle of custom integrations into N + M, letting any model use any tool that speaks it. - Agents fix agency. Wrap the model in a Reason, Act, Observe loop, framed as a policy pursuing a goal, so it can solve problems that take many steps. That's how a frozen model ends up answering a question it was never trained on, about an order placed last week. And the most useful skill is knowing when not to reach for each. If one retrieval answers the question, you don't need an agent. If the answer already lives in the model's weights, you don't need RAG. Each layer buys capability at the cost of complexity, so add them only when the problem demands it. --- Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents): - Sushant Waidande - Sai Krishna Pashapu (AI Developer, BTech, [LinkedIn](https://www.linkedin.com/in/sai-krishna-pashapu-997358283/?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents), [GitHub](https://github.com/saikrishna-pashapu?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents)) - Bob Boncella - Demet Yazici - Imran Khan --- If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend! Want to reach 8000+ ML engineers? Advertising Inquiries: [WORK WITH US 📈](https://tally.so/r/A7Dgdk?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=the-math-based-guide-to-rag-mcp-and-agents) Feedback or inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com). --- That’s it for this week’s issue of AI, but simple. See you next week! —AI, but simple team