Modern network optimizers

Optimizing multidimensional functions, such as neural networks, is a complex task. The gradient descent solution we presented in the first chapter is an elegant solution, though it has some limitations that we will highlight in the following section. Thankfully, researchers have been developing new generations of optimization algorithms, which we will also discuss.