The basic idea behind recurrent neural networks is the vectorization of data. If you look at figure Fixed sized inputs of neural networks, which represents a traditional neural network, each node in the network accepts a scalar value and generates another scalar value. Another way to view this architecture is that each layer in the network accepts a vector as its input and generates another vector as its output. Figure Neural network horizontally rolled up and figure Neural network vertically rolled up illustrate this representation:
Neural network horizontally rolled up
Neural network vertically rolled up
The figure Neural network vertically rolled up is a simple RNN representation, which is a one-to-one RNN; one input is mapped to one output using one hidden layer.
