Optimizing hyperparameters

There are many different ways of tuning hyperparameters. If we were to do this manually, we could put random variables into our parameters and see which one was the best. To do this, we could perform a grid-wise approach, where we map the possible options and put in some random tries and keep going down a route that seems to produce the best outcomes. We might use statistics or machine learning to help us determine what parameters can give us the best results. These different approaches have pros and cons, depending on the shape of the loss of the experiment. 

There are various machine learning libraries that can help us perform these types of common tasks easier. sklearn, for example, has a RandomizedSearchCV method that, given a set of parameters, will perform a search for the best model with the least loss. In this recipe, we will expand on the Classifying chemical sensors with decision trees recipe from Chapter 3, Machine Learning for IoT, and...