Cleaning and Processing Data

On rare occasions, you may receive data that is already clean, neat, and ready to use, but having an immaculate dataset just handed to you is the exception, not the rule. More often than not, while working as a data analyst, the datasets you receive will be messy, incomplete, and completely unusable without a little work. Trying to use jumbled data will only give you jumbled results. This chapter covers the most common issues you will come across and a few approaches to dealing with them.

Here, we will discuss the difference between duplicate data and redundant data, as well as what to do about it. Then, we will discuss why missing data is an issue and the different approaches you can take to deal with it. Briefly, we will cover invalid data, mismatched data, and data type validation. After that, we will discuss non-parametric data, what it is, and how to approach it. Finally, we will discuss outliers or data points that don’t seem to fit in with...