In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton published a milestone paper titled ImageNet Classification with Deep Convolutional Neural Networks https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf. The paper describes their use of neural networks to win the ImageNet competition of the same year, which we mentioned in Chapter 2, Neural Networks. At the end of their paper, they wrote the following:
"It is notable that our network's performance degrades if a single convolutional layer is removed. For example, removing any of the middle layers results in a loss of about 2% for the top-1 performance of the network. So the depth really is important for achieving our results."
They clearly mention the importance of the number of hidden layers present in deep networks. Krizheysky...
