Thinking about backpropagation and convnets

In this section we want to give an intuition behind backpropagation and convnets. For the sake of simplicity we will focus on an example of convolution with input X of size 3 × 3, one single filter W of size 2 × 2 with no padding, stride 1, and no dilation (see Chapter 5, Advanced Convolutional Neural Networks). The generalization is left as an exercise.

The standard convolution operation is represented in Figure 15. Simply put, the convolutional operation is the forward pass:

Figure 15: Forward pass for our convnet toy example

Following the intuition of Figure 15, we can now focus our attention to the backward pass for the current layer. The key assumption is that we receive a backpropagated signal as input, and we need to compute and . This computation is left as an exercise but please note that each weight in the filter contributes to each pixel in the output map or, in other words, any change in a weight of a...