Summary

In this chapter, we introduced two complementary topics – NLP and RNNs. We discussed the tokenization technique and the most popular tokenization algorithms – BPE, WordPiece, and Unigram. Then, we introduced the concept of word embedding vectors and the Word2Vec algorithm to produce them. We also discussed the n-gram LM, which provided us with a smooth transition to the topic of RNNs. There, we implemented a basic RNN example and introduced two of the most advanced RNN architectures – LSTM and GRU. Finally, we implemented a sentiment analysis model.

In the next chapter, we’ll supercharge our NLP potential by introducing the attention mechanism and transformers.