Introduction of DNNs

With the advent of big data processing infrastructure, GPU, and GP-GPU, we are now able to overcome the challenges with shallow neural networks, namely overfitting and vanishing gradient, using various activation functions and L1/L2 regularization techniques. Deep learning can work on large amounts of labeled and unlabeled data easily and efficiently.

As mentioned, deep learning is a class of machine learning wherein learning happens on multiple levels of neuron networks. The standard diagram depicting a DNN is shown in the following figure:

From the analysis of the previous figure, we can notice a remarkable analogy with the neural networks we have studied so far. We can then be quiet, unlike what it might look like, deep learning is simply an extension of the neural network. In this regard, most of what we have seen in the previous chapters is valid. In short, a DNN is a multilayer neural network that contains two or more hidden layers. Nothing very complicated here...