# Ensemble Models, Simply Explained #74 Oct 27, 2025 Foundation https://www.aibutsimple.com/p/ensemble-models-simply-explained Imagine you’re watching a boxing competition. The bell for the final round rings, and it goes to the judges’ scorecard. You notice that there is a panel of three judges. Why is it that there are usually at least three judges and never just one? Well, judges can be biased, tired, or make errors and miss important details. Accordingly, we have a panel of judges to form a collective decision. Combining their final scores, we get a final decision that is more balanced and more fair than using only one individual judge’s score. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/7aaa1641-7c1d-496d-88dd-aee745a759f2/Ensembling_Judge_Decisions__1_.png) In essence, this is exactly what ensembling does in machine learning. The individual judgments of multiple models to make decisions are all considered or combined in order to make one final, more accurate decision. Every AI model learns differently. One model could potentially be overfit, meaning it has memorized the training data too closely, while another might underfit, meaning it has oversimplified the true complexity of the data. - Want to learn more about overfitting/underfitting? [We go over this concept in another issue, linked here](https://www.aibutsimple.com/p/deep-learning-overfitting-underfitting-vanishing-gradient?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained). Combining their outputs, either by averaging, voting, or stacking them, means individual mistakes can potentially be cancelled out, to an extent—the overall performance will increase. Hence, just like the case in boxing when we can have judges to offer more than one perspective, models can also consider all outputs to make a final decision that is more accurate. Ensemble models can generalize better than singular models. Ensembling is effectively a method to reduce variance, which is a measure of how much uncertainty a specific model contains. By combining several models that have errors in different directions, these uncertainties are smoothed out, and a more stable prediction range is output. In simplest terms, ensembling does not make any individual model smarter, but it makes a collection of models less wrong together. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/453234bf-e384-4e18-b659-770a219be5e8/Average_between_models_ensembling__1_.png) In the above diagram, the red, blue, and green waves indicate noisy models, whereas the bold, black line showcases the weighted average of the three colored models. --- ### The AI Insights Every Decision Maker Needs [![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/b6c4339d-a80d-4f96-8aef-6aafa6a20425/Phone_4.png)](https://subscribe.thedeepview.com/?utm_campaign=XIAEVJHTGJ&utm_source=beehiiv&utm_medium=newsletter&_bhiiv=opp_0bc1bafb-0dc6-44de-8238-b4e714408ae0_12ba3285) You control budgets, manage pipelines, and make decisions, but you still have trouble keeping up with everything going on in AI. If that sounds like you, don’t worry, you’re not alone – and [The Deep View](https://subscribe.thedeepview.com/?utm_campaign=XIAEVJHTGJ&utm_source=beehiiv&utm_medium=newsletter&_bhiiv=opp_0bc1bafb-0dc6-44de-8238-b4e714408ae0_12ba3285) is here to help. This [free, 5-minute-long daily newsletter covers everything you need to know about AI](https://subscribe.thedeepview.com/?utm_campaign=XIAEVJHTGJ&utm_source=beehiiv&utm_medium=newsletter&_bhiiv=opp_0bc1bafb-0dc6-44de-8238-b4e714408ae0_12ba3285). The biggest developments, the most pressing issues, and how companies from Google and Meta to the hottest startups are using it to reshape their businesses… it’s all broken down for you each and every morning into easy-to-digest snippets. If you want to up your AI knowledge and stay on the forefront of the industry, [you can subscribe to The Deep View right here (it’s free!).](https://subscribe.thedeepview.com/?utm_campaign=XIAEVJHTGJ&utm_source=beehiiv&utm_medium=newsletter&_bhiiv=opp_0bc1bafb-0dc6-44de-8238-b4e714408ae0_12ba3285) --- ### Ensembling Methods Most ensembling methods fall into three major categories: Bagging, Boosting and Stacking. Each takes a slightly different approach in combining models. #### Bagging [Bagging (short for Bootstrap Aggregating)](https://en.wikipedia.org/wiki/Bootstrap_aggregating?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained) trains several models on different random slices of the dataset. Each model therefore forms a unique opinion, given it was trained on a unique dataset, causing all models to have learned slightly differently. All model predictions are then either averaged or voted on. Think of it like this: Instead of reading one in-depth article, you summarize five different, shorter ones. Each focuses on a slightly different main topic, but as a collective, reading them gives you a more well-rounded, less biased picture of the event. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/6591c3bf-9b30-45e0-923d-d5126e432e32/Ensemble_Modelling__Individual_to_Large.png) An example of bagging is the [Random Forest](https://en.wikipedia.org/wiki/Random_forest?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained), where many decision trees vote together. One tree might get it wrong, but the chance an entire forest does is significantly lower. This means the overall average performance will increase most of the time. #### Boosting [Boosting](https://en.wikipedia.org/wiki/Boosting_(machine_learning)?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained), on the other hand, does not train models independently, but instead trains them sequentially. Each new model trained is focused on fixing the mistakes of the previous ones. Think of each model like a teacher that is teaching you a skill. The first teacher would teach you the basics. The second teacher can build on your knowledge by focusing on helping you correct your weak points. The third teacher would continue to build on this progress in the same manner. By the end, the variety in perspectives received makes you better at that skill than if you’d just stayed with one teacher, as they are indirectly correcting each other’s shortcomings. In the diagram below, the red portions represent areas of the model that underperform and that get corrected by the next model. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/2a7b16c4-f58e-4963-8ccd-80370bcf81fa/Boosting__Ensemble_Models.png) Popular boosting algorithms include [AdaBoost](https://en.wikipedia.org/wiki/AdaBoost?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained), [Gradient Boosting](https://en.wikipedia.org/wiki/Gradient_boosting?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained) and [XGBoost](https://en.wikipedia.org/wiki/XGBoost?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained). All of these algorithms are widely used in data science competitions and real-world applications. --- ### [Level Up Your Career With AI, But Simple PRO](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained) ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/281697a8-c915-435c-a68b-43374b671698/aibutsimple_academy.png) Supporting us by purchasing a PRO membership will get you: - Exclusive Hands-on Tutorials With Code - Full Math Explanations and Beautiful Visualizations - Ad-Free Reading for an Uninterrupted Learning Experience [Break into AI or sharpen your edge by joining today!](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained) --- #### Stacking Finally, there is Stacking (short for Stacked Generalization), which is essentially to have another model consider all models and to decide which ones are to be used the most. Various models can be trained, such as decision trees or neural networks, and their outputs are fed into a final model that learns how to combine them. It does so by determining how much each base model’s prediction is correlated to the true values and implicitly assigns weights to them based on performance, similar to any [supervised learning](https://www.aibutsimple.com/p/supervised-machine-learning-simplified?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained) process. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/e3a48277-87a0-4d52-b3d3-8534ca89008c/Stacking_Ensembling_Method.png) Above, the red numbers indicate the weighting that each model is given in the creation of the final model, representing their corresponding influence in the final output. ### Ensembling in Modern AI Ensembling is not just a theoretical concept; it powers a majority of the AI tools that influence our lives today. For instance, when any video-streaming platform recommends something to watch for you, it combines multiple recommendation systems. This includes what you’ve previously watched, how long you watched it for, what similar users liked, etc. This combination of different metrics allows for more accurate predictions. Naturally, however, more models are not always ideal. Ensembling can get more expensive, both in time and computing power. If each model takes hours to train, it can very quickly become wasteful. Furthermore, ensembles can hide individual weaknesses. If one model is strongly biased, its vote will still influence the final result. ![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/1cef92fa-abaf-4f04-be80-2f60015f490e/Ensembling_8_different_models.png) Diversity in models matters—considering different perspectives leads to fairer outcomes. In the diagram above, the variables in red represent the corresponding weight that every model is given in the creation of the final, ensemble model. Today, almost all top AI models use some form of ensembling. It acts as a reminder that intelligence is not about perfection, but collaboration. The future of AI lies not in building one genius model, but many complementary ones acting together. The next time the perfect video is recommended to you, remember it’s not just one smart algorithm, but an entire ensemble quietly collaborating to understand you best. --- Here’s a special thanks to our [biggest supporters](https://www.aibutsimple.com/upgrade?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained): - Sushant Waidande - Sai Krishna Pashapu - Bob Boncella --- If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend! Want to reach 50000+ ML engineers? Let’s work together: [WORK WITH US 📈](https://www.passionfroot.me/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained) Feedback, inquiries? Send us an email at [edwin@aibutsimple.com](mailto:edwin@aibutsimple.com). If you like, you can also donate to our team to push out better newsletters every week! [Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=ensemble-models-simply-explained) --- That’s it for this week’s issue of AI, but simple. See you next week! —AI, but simple team