Supervised ML: Decision Trees

Hello from the AI, but simple team! If you enjoy our content, consider supporting us so we can keep doing what we do.

Our newsletter is no longer sustainable to run at no cost, so we’re relying on different measures to cover operational expenses. Thanks again for reading!

Supervised ML: Decision Trees

AI, But Simple Issue #51

Imagine you are planning a weekend picnic. You might wonder if it’s sunny and if it’s too hot to stay outdoors. Perhaps, if it’s cloudy, you might wonder if it’s windy or raining. You go through a sequence of questions, and it guides you towards a final decision—whether to go for a picnic or to stay home.

By breaking more complex decisions into a series of simpler questions, decision trees provide an intuitive and mathematical way to make predictions.

Decision trees are foundational models used in supervised learning for regression and classification. They are also referred to as CARTs, the classification and regression tree algorithm.

Get Your Free ChatGPT Productivity Bundle

Mindstream brings you 5 essential resources to master ChatGPT at work. This free bundle includes decision flowcharts, prompt templates, and our 2025 guide to AI productivity.

Our team of AI experts has packaged the most actionable ChatGPT hacks that are actually working for top marketers and founders. Save hours each week with these proven workflows.

It's completely free when you subscribe to our daily AI newsletter.

Subscribe to get instant access

Decision trees are categorized based on the type of target variable they predict.

Classification trees are used when the target variable is categorical—“Will it rain?” The resulting prediction will be a class.

Following the same logic, regression trees are used when the target variable is numerical—“How much rainfall?” The prediction will be a numerical value.

Both types of trees are built the same way but handle the output differently: classification trees use majority voting to predict classes, while regression trees use the mean or median of all the probable values.

The components of a decision tree form a tree-like structure that makes them simple to interpret:

• The root node represents the entire dataset; it’s the starting point of the decision process. Decision nodes are tests on features (“Is it raining?”), while branches represent the outcomes of the tests (“Yes” or “No”). The branches are represented as arrows in the diagram below.

• Finally, leaf nodes are the final predictions or decisions, whether they are classes or numerical values.

Why Decision Trees?

The decision tree’s structure is similar to human decision-making. Comparable to flowcharts, their tree structure is intuitive and can be visualized as a simple diagram with straightforward logic, making them easy to learn and implement.

• Decision trees are a popular foundational machine learning algorithm and, in most cases, are used to introduce machine learning concepts to those who don’t already have an advanced technical background.

They require no feature scaling or normalization, unlike algorithms like SVMs or neural networks, which require more data processing.

Furthermore, they are effective for more complex non-linear tasks and handle outliers decently well for their simplicity.