Web scraping fundamentals

Web scraping, as used in this book, is the process of extracting information from an HTML-structured page that is intended to be viewed by a human and not consumed programmatically. Some services provide an API that is efficient for programmatic use, but some websites only provide their information in HTML pages. These web scraping examples demonstrate various ways of extracting information from HTML. We'll look at basic string matching, then regular expressions, and then a powerful package named goquery, for web scraping.

Finding strings in HTTP responses with the strings package

To get started, let's look at making a basic HTTP request and searching for a string using the standard library...