Modeling the topics of online news stories

To see how topic models perform on real data, we will look at two datasets containing articles originating from BBC News during the period of 2004-2005. The first dataset, which we will refer to as the BBC dataset, contains 2,225 articles that have been grouped into five topics. These are business, entertainment, politics, sports, and technology.

The second dataset, which we will call the BBCSports dataset, contains 737 articles only on sports. These are also grouped into five categories according to the type of sport being described. The five sports in question are athletics, cricket, football, rugby, and tennis. Our objective will be to see if we can build topic models for each of these two datasets that will group together articles from the same major topic.

Note

Both BBC datasets were presented in a paper by D. Greene and P. Cunningham, entitled Producing Accurate Interpretable Clusters from High-Dimensional Data and published in the proceedings...