Finding the most popular movie

Let's start by reviewing the data format of the MovieLens dataset, the u.data file.

As you might recall, the u.data file on each line, consists of a user ID, a movie ID, a rating, and a timestamp. Each line says, "this user watched this movie, gave it this rating, and did it at this time":

Our task is to just figure out which movie was watched most often or which movie ID appears most frequently in the entire dataset. This isn't a very hard thing to do; in fact, if you want to go give it a crack yourself, feel free. In this section we'll take a look at the implementation that I came up with, get that to run, and see what we come up with

Examining the popular-movies script

In the download package for this book, you'll find a popular-movies Python script. Download that, put it in your SparkCourse folder, and open it up. We will print out a list of all of the movies and the number of times they appear, and then sort them based on how often they appear. It's a pretty...