Modeling and evaluation

Modeling will be broken in two distinct parts. The first will focus on word frequency and correlation and culminate in the building of a topic model. In the next portion, we will examine many different quantitative techniques by utilizing the power of the qdap package in order to compare two different speeches.

Word frequency and topic models

As we have everything set up in the document-term matrix, we can move on to exploring word frequencies by creating an object with the column sums, sorted in descending order. It is necessary to use as.matrix() in the code to sum the columns. The default order is ascending, so putting - in front of freq will change it to descending:

> freq = colSums(as.matrix(dtm))

> ord = order(-freq)

We will examine head and tail of the object with the following code:

> freq[head(ord)]
american     year      job     work  america      new 
     243      241      212      195      187      177 

> freq[tail(ord)]
      voic     welcom...