Supervised ML: Decision Trees

Imagine you are planning a weekend picnic. You might wonder if it’s sunny and if it’s too hot to stay outdoors. Perhaps, if it’s cloudy, you might wonder if it’s windy or raining. You go through a sequence of questions, and it guides you towards a final decision—whether to go for a picnic or to stay home.

By breaking more complex decisions into a series of simpler questions, decision trees provide an intuitive and mathematical way to make predictions.

Decision trees are foundational models used in supervised learning for regression and classification. They are also referred to as CARTs, the classification and regression tree algorithm.

Decision trees are categorized based on the type of target variable they predict.

Classification trees are used when the target variable is categorical—“Will it rain?” The resulting prediction will be a class.

Following the same logic, regression trees are used when the target variable is numerical—“How much rainfall?” The prediction will be a numerical value.

Both types of trees are built the same way but handle the output differently: classification trees use majority voting to predict classes, while regression trees use the mean or median of all the probable values.

The components of a decision tree form a tree-like structure that makes them simple to interpret:

• The root node represents the entire dataset; it’s the starting point of the decision process. Decision nodes are tests on features (“Is it raining?”), while branches represent the outcomes of the tests (“Yes” or “No”). The branches are represented as arrows in the diagram below.

• Finally, leaf nodes are the final predictions or decisions, whether they are classes or numerical values.