Chapter 5 - Improvements to the RNN

1. A Long Short-Term Memory (LSTM) cell is a variant of an RNN that resolves the vanishing gradient problem by using a special structure called gates. Gates keep the information in the memory as long as it is required. They learn what information to keep and what information to discard from the memory.
2. LSTM consists of three types of gates, namely, the forget gate, the input gate, and the output gate. The forget gate is responsible for deciding what information should be removed from the cell state (memory). The input gate is responsible for deciding what information should be stored in the cell state. The output gate is responsible for deciding what information should be taken from the cell state to give as an output.
3. The cell state is also called internal memory where all the information will be stored.
4. While backpropagating the LSTM network...