Menu

Understanding Decision Trees

Understanding Decision Trees

The foundation of XGBoost is the decision tree.

A decision tree makes predictions by repeatedly splitting data into smaller groups based on feature values.

For example:

Does the house have more than 6 rooms?

|

Yes / No

|

Further splits

|

Predicted Price

Each split helps separate houses with different price ranges.

Decision trees are easy to understand and can capture non-linear relationships between variables.

However, a single tree may not always produce the best predictions.

This is where boosting becomes useful.