Measuring progress with loss

When we discussed the areas of classification and regression, we outlined a few measures to measure and quantify the performance of our models relative to one another. When it came to classification, we used precision and accuracy, whereas, in regression, we used MAE and MSE. Within the confines of deep learning, we will use a metric known as loss. The loss of a neural network is simply a measure of the cost that's incurred from making an incorrect prediction. Take, for example, a simple neural network with three input values and a single output value:

Figure 8.16 – A neural network showing input and output values

In this case, we have the values [2.3, 3.3, 1.2] being used as input values to the model, with a predicted value of 0.2 relative to the actual value of 1.0. We can demonstrate the loss as follows:

In this function, is the predicted value, while is the actual...