Bagging Explained

Bagging, short for Bootstrap Aggregating, trains multiple versions of the same model on different random subsets of the training data, then combines their results.

Each subset is created by sampling the original data with replacement, meaning some data points may appear multiple times in one subset, while others may not appear at all.

Once all models are trained separately, their predictions are combined, usually through voting for classification or averaging for regression, to produce the final output.

Applications

• Random forests use bagging as their core technique, training many decision trees.
• Bagging is applied in medical diagnosis models to reduce prediction variance.
• It is used in financial forecasting to stabilize predictions across volatile data.

Examples

• A random forest predicting loan default risk by combining hundreds of individual decision trees.
• A bagged model averaging predictions from multiple regression trees to estimate house prices more reliably.