Summary

In this chapter, we started off with a discussion on overfitting and underfitting and how these can affect the performance of a model on unseen data. The chapter looked at ensemble modeling as a solution for these and went on to discuss different ensemble methods that could be used, and how they could decrease the overall bias or variance encountered when making predictions.

We first discussed bagging algorithms and introduced the concept of bootstrapping. Then, we looked at Random Forest as a classic example of a Bagged ensemble and solved exercises that involved building a bagging classifier and Random Forest classifier on the previously seen Titanic dataset.

We then moved on to discussing boosting algorithms, how they successfully reduce bias in the system, and gained an understanding of how to implement adaptive boosting and gradient boosting. The last ensemble method we discussed was stacking, which, as we saw from the exercise, gave us the best accuracy score of all the ensemble...