Fully connected layers

We have already learned about fully connected layers in Chapter 2, Dive Deep into Deep Neural Networks. Having fully connected layers simply means that all the nodes in one layer are connected to the outputs of the next layers. The output of the fully connected layer is a class of probabilities, where each class is assigned a probability. All probabilities must sum up to 1. The activation function used at the output of the layer is called the softmax function.