Hyperparameters in machine learning

In Chapter 7, Enhancing Machine Learning Models using Feature Selection, we described supervised learning as the programmatic process of adjusting (or tuning) the internal parameters of a model to produce the desired outputs in response to given inputs. To make this happen, each type of supervised learning model is accompanied by a learning algorithm that iteratively adjusts its internal parameters during the learning (or training) phase.

However, most models have another set of parameters that are set before the learning takes place. These are called hyperparameters, and affect the way the learning is done. The following image illustrates the two types of parameters:

Hyperparameter tuning of a machine learning model

Usually, the hyperparameters have default values that will take effect if we don't specifically set them. For example,...