Understanding outliers and trends

Finding outliers begins by looking at the distribution curve but requires additional techniques that we will walk through together. Additionally, don't underestimate the need for soft skills where you must reach out to others to better understand why an outlier exists in your data. An outlier is commonly known as one or more data values that are significantly different than the rest of the data. Spotting outliers in data is easy depending on the data visualization used, but in many cases, especially when data volumes are very large, they can be obscured when data is aggregated. If you recall from Chapter 7, Exploring Cleaning, Refining, and Blending Datasets, we worked with hits created by a user for a website. A good example of obscuring outliers is when those user hits are aggregated by date. If a specific user has 1,000 hits per day when the average is 2, it would be difficult to identify that outlier user after the data was aggregated...