Performing abstractive summarization

Abstractive summarization generates novel sentences by rephrasing the reference and introducing new text. This task is quite challenging, and for this reason, more sophisticated methods are required. This section adopts a step-by-step approach to present pertinent concepts and techniques. Ultimately, we glue all the pieces together in a state-of-the-art model for abstractive summarization. Let’s begin with the first concept.

Introducing the attention mechanism

In Chapter 6, Teaching Machines to Translate, we presented an encoder-decoder seq2seq architecture suitable for translating sentences from a source language to a target one. A key characteristic of the whole pipeline is that the complete input is encoded in a context vector used by the decoder to produce a translation. In actual human communications, we tend to listen to the whole sentence before responding. Intuitively, the context vector represents this process; it crams the...