Training a vanilla neural network

To understand how to train a vanilla neural network, we will go through the task of predicting the label of a digit in the MNIST dataset, which is a popular dataset of images of digits (one digit per image) and the corresponding label of the digit that is contained in the image.

Getting ready

Training a neural network is done in the following steps:

1. Import the relevant packages and datasets
2. Preprocess the targets (convert them into one-hot encoded vectors) so that we can perform optimization on top of them:
   • We shall be minimizing categorical cross entropy loss
3. Create train and test datasets:
   • We have the train dataset so that we create a model based on it
   • The test dataset is not seen...