Introduction to RNNs

To understand RNNs, we have to understand the basics of feedforward neural networks. You can refer to Chapter 3, Optimization for Neural Networks, for details on feedforward networks. Both feedforward and recurrent neural networks are identified from the way they process the information or features through a series of mathematical operations performed at the various nodes of the network. One feeds information straight through (never touching a given node twice), the other cycles it through a loop.

A feedforward neural network is trained on image data until it minimizes the loss or error while predicting or classifying the categories for image types. With the trained set of hyper parameters or weights, the neural network can classify data it has never seen before. A trained feedforward neural network can be shown any random collection of images and the first image it classifies will not alter how it classifies the other images.

In a nutshell, these networks have no notion...