CNN architecture

In multilayer networks, such as MLP or DBN, the outputs of all neurons of the input layer are connected to each neuron in the hidden layer, so the output will again act as the input to the fully-connected layer. In CNN networks, the connection scheme that defines the convolutional layer is significantly different. The convolutional layer is the main type of layer in CNN, where each neuron is connected to a certain region of the input area called the receptive field.

In a typical CNN architecture, a few convolutional layers are connected in a cascade style, where each layer is followed by a rectified linear unit (ReLU) layer, then a pooling layer, then a few more convolutional layers (+ReLU), then another pooling layer, and so on.

The output from each convolution layer is a set of objects called feature maps that are generated by a single kernel filter. The feature maps can then be used to define a new input to the next layer. Each neuron in a CNN network produces an output...