In the previous recipe, Investigating advanced classifiers, we introduced a few examples of ensemble models. They used multiple decision trees (each model in a slightly different way) to build a better model. The goal was to reduce the overall bias and/or variance. Similarly, stacking is a technique that combines multiple estimators. It is a very powerful and popular technique, used in many competitions.
We provide a high-level overview of the characteristics:
- The models used as base learners do not need to be homogeneous—we can use a combination of different estimators. For example, we can use a decision tree, a k-nearest neighbors classifier, and logistic regression.
- Stacking uses a meta learner (model) to combine the predictions of the base learners and create the final prediction.
- Stacking can be extended to multiple levels...
