Optimization with stochastic gradient descent

Stochastic gradient descent (SGD), in contrast to batch gradient descent, performs a parameter update for each training example, x⁽ⁱ⁾ and label y⁽ⁱ⁾:

Θ = Θ - η∇_Θj(Θ, x⁽ⁱ⁾, y⁽ⁱ⁾)

Getting ready

Make sure that the preceding common code list is added before the main code snippet in the following codes:

How to do it...

Create a sequential model with the appropriate network topology:

• Input layer with shape (*, 784), and an output of (*, 512)
• Hidden layer with...