Comparing XGBoost with other ensemble decision tree models
So far in this chapter, as you’ve learned how XGBoost works, you’ve also seen how it compares to the gradient-boosted tree algorithm in scikit-learn or in R. The authors of the XGBoost paper have added features that enhance the existing algorithm and make it faster and more accurate in its predictions, or at least less prone to overfitting. Now, let’s wrap up this chapter by comparing XGBoost with random forest models.
XGBoost compared with random forest models
In XGBoost, as with any gradient-boosted tree model, the decision trees are built iteratively one after another, to improve the results from the previously built trees. As a result, these decision trees are not independent. Figure 3.5 shows how a gradient-boosted tree algorithm uses the results from the previous trees and the dataset to improve the model accuracy as measured by the training fit metrics in an iterative manner.
