Setting XGBoost regression hyperparameters
Hyperparameters are settings that affect model training. In this section, you will learn about the hyperparameters for XGBoost, what their default settings are, and how to change these settings. This section will cover regression in particular. Typically, you will use experimentation to decide on the best hyperparameter values to use when creating a model for the data. This is because the settings are dataset-dependent. There are software packages such as SigOpt that help keep track of your hyperparameter experiments and automate the process. These packages are outside the scope of this book.
The following table (Table 5.2) lists the hyperparameters you should consider modifying to suit your dataset. This is a subset of the available XGBoost learning parameters that are listed in the XGBoost documentation: https://xgboost.readthedocs.io/en/stable/parameter.html. XGBoost is highly configurable: