The performance of the BERTSUM model

The researchers of BERTSUM have used the CNN/DailyMail news dataset. The CNN/DailyMail dataset consists of news articles along with their highlights. We split the CNN/DailyMail news dataset into train and test sets. We train the model using the train set and evaluate it on the test set.

The following shows the ROUGE score of an extractive summarization task using BERTSUM with a classifier, a transformer, and LSTM. We can observe that BERTSUM with the transformer performs slightly better than the others:

Figure 6.14 – ROUGE score of an extractive summarization task using BERTSUM

The following shows the ROUGE score of the abstractive summarization task using BERTSUMABS:

Figure 6.15 – ROUGE score of the abstractive summarization task using BERTSUMABS

Thus, we have learned how to fine-tune the BERT model for abstractive and extractive summarization tasks. In the next section, we will see how to train the BERTSUM model.