# AI Hallucinations, Simply Explained #47 Apr 21, 2025 Safety https://www.aibutsimple.com/p/ai-hallucinations-simply-explained-7a8c Picture this: you ask a chatbot about a historical event, and it confidently gives you details that seem accurate—only to discover later that these details are completely made up. You have just encountered what we call an AI hallucination. In artificial intelligence, a hallucination refers to a situation where a neural network produces outputs that seem coherent and reasonable but are factually incorrect or made up. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/fbfff84e-9e2c-4175-9f28-2bf0f53045f2/Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Add_a_heading__720_x_600_px_.png) For example, a language model might claim a famous figure was born in the wrong city, or an image generation model might produce an image that doesn’t match the provided prompt. Unlike human error, which can be random or accidental, AI hallucinations are generated based on patterns the model has learned. The model does not even recognize that its output is wrong; it simply generates what seems statistically likely from its past training data. Hallucinations are particularly frequent in generative models, machine learning models that generate new content based on past training examples. Some common generative models include: - [Variational Autoencoders (VAEs)](https://www.aibutsimple.com/p/a-visual-guide-to-variational-autoencoders-vaes?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained), autoencoder-based models that generate new similar data by learning a probabilistic latent space. - [Generative Adversarial Networks (GANs)](https://www.aibutsimple.com/p/generative-adversarial-networks-simplified?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained), a model that balances a generator against a discriminator to create realistic outputs. - [Transformers](https://www.aibutsimple.com/p/transformers-and-the-attention-mechanism?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained) (such as GPT), models that generate sequences, like text, by predicting the next token/word based on context. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/1d85963e-aa0b-45fc-831c-b95969f21b59/Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Add_a_heading__720_x_600_px___1_.png) Hallucinations are happening everywhere and can occur for many reasons, from model design to pure chance. Since many models don’t have sophisticated reasoning techniques, they present their answers with high confidence. --- Big thanks to our partners for keeping this newsletter free. If you have a second, clicking the ad below helps us a ton—and who knows, you might find something you’ll enjoy. ### Start learning AI in 2025 [![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/3063de3b-8243-4e98-bb10-581679f4b033/Learn_AI_in_3_Minutes_Beehive__1_.jpg)](https://www.superhuman.ai/c/confirmation?magiclink_subscription) Keeping up with AI is hard – we get it! That’s why over 1M professionals read [Superhuman AI](https://www.superhuman.ai/c/confirmation?magiclink_subscription) to stay ahead. - Get daily AI news, tools, and tutorials - Learn new AI skills you can use at work in 3 mins a day - Become 10X more productive [Sign up and start mastering AI](https://www.superhuman.ai/c/confirmation?magiclink_subscription) --- ### Why Do Hallucinations Happen? #### Design of Generative Models Generative models such as GPT do not know and enforce factual information; they simply model probabilities given certain inputs. What do we mean by this? Well, most generative models use probability distributions to generate new and accurate data. For instance, transformers, a type of generative model widely used in natural language processing (NLP), are given input sequences, which they tokenize, embed, and pass through the transformer body. The outputs of the body are linearly transformed, then passed through a Softmax activation function to generate probabilities for each token (that indicate how likely a token is to be the next word). The end result is just a probability distribution that the model can sample from. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0dbdb0d0-7ceb-4d28-9212-0b208eee7fff/Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Add_a_heading__720_x_600_px___2_.png) If we ask a transformer-based model, "What is the capital of France?”, the model doesn’t retrieve a stored answer. It computes the probability of “Paris” being the correct token given the input sentence. In other words, the model is computing the conditional probability of the token “Paris” given the question asking it for the capital of France: P(“Paris” | “What is the capital of France?”) Based on its training, “Paris” has a high probability, so it can be selected. However, for less clear or less known questions, the model might assign small non-zero probabilities to incorrect answers (for instance, “London”) because it’s seen similar patterns in unrelated contexts. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/b865ebe4-51d9-44b0-a9f6-b27a8c60cec9/cap_of_france.png) If the model samples a low-probability (but possible) token, it becomes a hallucination. Even if the model has a 1% chance of choosing “London” due to some noisy training data, there is always a chance that it still may be sampled. --- ### Your team just got supercharged. [Moxby](https://www.vpdae.com/redirect/hi9rftkzemjmkxut8ryrhz6ifr9?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained) gives every employee an AI agent that executes tasks, learns new skills, and even writes its own code to improve—no prompt engineering required. Each Moxby agent is trained on its user’s workflow and built to deliver. From building internal tools to running full campaigns, it learns fast, adapts, and becomes a true digital teammate. [Join the Waitlist](https://www.vpdae.com/redirect/hi9rftkzemjmkxut8ryrhz6ifr9?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained) --- Transformers in particular are also more prone to hallucinations due to their design and architecture. When transformers generate responses, they do so by using a method called next-word prediction. In next-word prediction, the model generates words one-by-one sequentially until the output is generated. The problem lies in the fact that generating one token at a time allows early errors to affect the entire generated sequence, influencing probabilities until the model ultimately hallucinates. Moreover, next-word prediction encourages the model to generate text even when it lacks sufficient information. A model might still generate a response that sounds convincing but isn’t accurate—it can produce a response based on related patterns that it learned rather than facts. #### Decoding Strategies Another reason models hallucinate is based on how they sample from the probability distribution. During inference, transformers use specific strategies to select tokens to output from the probability distribution. Some common strategies include: - Greedy decoding, which is to pick the highest-probability token at each step. This leads to safe inputs most of the time but can be repetitive. - Randomly sampling from the probability distribution, choosing tokens based on their probabilities. This allows for more creative (variable) output but carries the risk of low-probability errors. - Top-k or top-p sampling, which are the most widely used sampling methods. In top-k or top-p sampling, we sample from the top k tokens or tokens with a cumulative probability of p. This method balances variability and coherence in output but is still prone to sampling wrong tokens (causing hallucinations). For instance, in top-p sampling, if p = 0.7, the model samples from tokens that make up 70% of the cumulative probability. An incorrect low-probability token might sneak in depending on the p or k value, causing a hallucination. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/cabaefd6-97f0-498f-b7f7-9de85fa6be38/asdasdasdasdasda.png) #### Training Data Limitations Datasets are finite, and they can’t cover every possible fact or context. The limited nature of training data contributes to model hallucinations. If the data is incomplete and lacks information about certain topics, when posed a question, the model creates an answer based on patterns of other related topics or similarly structured prompts. Asked about a rare historical event, it might invent details. Today, more models are pre-trained on the internet than ever before. Data from the internet is accessible and cheap, but it’s also noisy. Internet-sourced data often contains errors, contradictions, and fiction, which can be replicated by the model in the form of hallucinations. - Additionally, large datasets are often collected using automated methods (like web scraping) that are not regulated properly and are inconsistent. For instance, if GPT encounters the sentence “London is the capital of France” while training on a satirical article, it might assign some probability to that false statement. Bias and ambiguity can also be found in a lot of data, overrepresenting certain perspectives, leading the model to favor them incorrectly. #### Generalization and Reasoning Neural networks compress huge amounts of data into their parameters. A transformer with billions of parameters can’t store every fact explicitly, so it generalizes patterns to perform well on most things. However, this generalization can also backfire. For example, it might learn that “birds can fly” but not that penguins are birds that cannot fly. This is an example of an overgeneralization, when a model oversteps the generalization process, usually if the input is not following a known pattern. In addition, transformers don’t reason like human beings. They simply predict tokens based on statistical patterns; they don’t have a strong logic sense. To illustrate this, one of the biggest misconceptions about transformers is how they answer arithmetic examples. If you prompt a model with “What is 2 + 3?”, the model doesn’t compute 2 + 3. It predicts “5” because it’s seen similar sequences. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/cedcfe98-de03-406f-aa12-59b5e7f4867e/Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Copy_of_Add_a_heading__720_x_600_px_.png) So if the context is noisy and unfavorable conditions pile up, that is how it might hallucinate “6” or some other incorrect answer. Models also lack a mechanism to verify facts against a larger database (what we call a world model) and therefore cannot cross-check claims—they cannot determine if their output physically makes sense. ### How Do We Reduce Model Hallucinations? To mitigate AI hallucinations, we should aim for better training data. Curating high-quality and diverse datasets is expensive but does minimize noise and gaps. We can also fine-tune the model on specific tasks to reinforce correct patterns, using techniques like reinforcement learning from human feedback (RLHF). ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/77ec5372-af23-41e1-8952-731424d8f5e8/image.png) A technique that has grown in popularity in these past years is Retrieval-Augmented Generation (RAG), where we pair the model with a database of facts to ensure outputs are in verified information. And perhaps the least exciting of all, training the model to express uncertainty (outputting “I’m not sure”) when probabilities are equal and ambiguous. --- Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained): Sushant Waidande --- If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend! Want to reach 50000+ ML engineers? Let’s work together: [WORK WITH US 📈](https://www.passionfroot.me/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained) Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com). If you like, you can also donate to our team to push out better newsletters every week! [Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ai-hallucinations-simply-explained) --- That’s it for this week’s issue of AI, but simple. See you next week! —AI, but simple team