6. Text Summarization and Text Generation

Activity 11: Summarizing a Downloaded Page Using the Gensim Text Summarizer

Solution

Let's summarize a downloaded page with the help of the Gensim text summarizer. Follow these steps to implement this activity:

1. Open a Jupyter notebook.
2. Insert a new cell and add the following code to import the necessary libraries:

   import warnings
   warnings.filterwarnings('ignore')
   from gensim.summarization import summarize
   import requests

3. The following code uses the requests library to get the Why Click page. After getting the page, we change the encoding to utf-8 in order to properly decode some of the content on the page. Then, we use BeautifulSoup to find the text content of the div with the ID #why-click. This div contains the main text of the why-click page:

   from bs4 import BeautifulSoup 
   r = requests.get('https://click.palletsprojects.com/en/7.x/why/')
   r.encoding = 'utf-8'
   soup = BeautifulSoup(r.text...