Introduction to Beautiful Soup 4 and Web Page Parsing

The ability to read and understand web pages is one of paramount interest for a person collecting and formatting data. For example, consider the task of gathering data about movies and then formatting it for a downstream system. Data for the movies is best obtained by the websites such as IMDB and that data does not come pre-packaged in nice forms(CSV, JSON< and so on), so you need to know how to download and read web page.

Furthermore, you also need to be equipped with the knowledge of the structure of a web page so that you can design a system that can search for (query) a particular piece of information from a whole web page and get the value of it. This involves understanding the grammar of markup languages and being able to write something that can parse them. Doing this, and keeping all the edge cases in mind, for something like HTML is already incredibly complex, and if you extend the scope of the bespoke markup language to include...