Chapter 6, Exploring BERTSUM for Text Summarization

1. In extractive summarization, we create a summary from a given text by just extracting the important sentences. In abstractive summarization, given a text, we will create a summary by re-expressing the given text using different words, holding only the essential meaning of the given text.
2. Interval segment embedding is used to distinguish between the multiple given sentences. With interval segment embedding, we map the tokens of the sentence occurring in the odd index to and we map the tokens of the sentence occurring in the even index to .
3. To perform abstractive summarization, we use the transformer model with encoder-decoder architecture. We feed the input text to the encoder and the encoder returns the representation of the given input text. In the encoder-decoder architecture, we use the pre-trained BERTSUM as an encoder. So, the pre-trained BERTSUM model will generate a meaningful representation and the decoder uses this representation...