Chapter 1: Introduction to Natural Language Processing

Activity 1: Generating word embeddings from a corpus using Word2Vec.

Solution:

1. Upload the text corpus from the link aforementioned.
2. Import the word2vec from gensim models

   from gensim.models import word2vec

3. Store the corpus in a variable.

   sentences = word2vec.Text8Corpus('text8')

4. Fit the word2vec model on the corpus.

   model = word2vec.Word2Vec(sentences, size = 200)

5. Find the most similar word to 'man'.

   model.most_similar(['man'])

   The output is as follows:

   

   Figure 1.29: Output for similar word embeddings

6. 'Father' is to 'girl', 'x' is to boy. Find the top 3 words for x.

   model.most_similar(['girl', 'father'], ['boy'], topn=3)

   The output is as follows:

Figure 1.30: Output for top three words for 'x'