Random forests

As we discussed when talking about trees, one of their advantages is their simplicity; but that is also the cause of their problems—their performance is often worse than other models, especially if the tree is small, in which case we will have what is known as a weak predictor. By the end of the 1980s, two researchers, Kearns and Valiant (1988, 1989), posted the question "Can a set of weak learners create a single strong learner?" This question gave rise to a lot of research on what is known as Ensemble methods or Ensemble Learning. The core idea of Ensemble Learning is simple—instead of using just one model to make predictions, use many individual models and combine their predictions. This simple idea has been one of the keys in the success of machine learning in producing very accurate models. Ensemble Learning is, of course, a whole sub...