Cleaning, refining, and purifying data using Python

Data quality is highly important for any data analysis and analytics. In many cases, you will not understand how good or bad the data quality is until you start working with it. I would define good-quality data as information that is well structured, defined, and consistent, where almost all of the values in each field are defined as expected. In my experience, data warehouses will have high-quality data because it has been reported on across the organization. In my experience, bad data quality occurs where a lack of transparency exists against the data source. Bad data quality examples are a lack of conformity and inconsistency in the expected data type or any consistent pattern of values in delimited datasets. To help to solve these data quality issues, you can begin to understand your data with the concepts and questions we covered in Chapter 1, Fundamentals of Data Analysis, with Know Your Data (KYD). Since the quality of...