Building a chatbot using sequence-to-sequence neural networks with attention
The easiest way to illustrate exactly how to implement attention within our neural network is to work through an example. We will now go through the steps required to build a chatbot from scratch using a sequence-to-sequence model with an attention framework applied.
As with all of our other NLP models, our first step is to obtain and process a dataset to use to train our model.
Acquiring our dataset
To train our chatbot, we need a dataset of conversations by which our model can learn how to respond. Our chatbot will take a line of human-entered input and respond to it with a generated sentence. Therefore, an ideal dataset would consist of a number of lines of dialogue with appropriate responses. The perfect dataset for a task such as this would be actual chat logs from conversations between two human users. Unfortunately, this data consists of private information and is very hard to come by within...
