Understanding and applying learning rate schedulers

In Chapter 12, Introduction to Time Series, Sequences, and Predictions. we built a DNN that achieved a mean absolute error (MAE) of 4.5. While this result was much better than our basic statistical methods, our next line of thought was how we could improve the performance of our DNN. One way of doing this is by finding the optimal learning rate. In Chapter 7, Image Classification with Convolutional Neural Networks, we discussed the important role of the learning rate in our modeling process as it controls the optimization process. Manually updating the learning rate can be a laborious process as the challenge lies in pinpointing what value works best. To have better control over the learning process, we apply a learning rate scheduler that adapts the learning rates based on defined criteria such as the number of epochs. With the aid of a LearningRateScheduler callback from TensorFlow, we can dynamically adjust the learning rate during...