Importing structured data using a well-defined schema

As seen in the previous chapter, Ingesting Data from Structured and Unstructured Databases, structured data has a standard format presented in rows and columns and is often stored inside a database.

Due to its format, the application of a DataFrame schema tends to be less complex and has several benefits, such as ensuring the ingested information is the same as the data source or follows a rule.

In this recipe, we will ingest data from a structured file such as a CSV file and apply a DataFrame schema to understand better how it is used in a real-world scenario.

Getting ready

This exercise requires the listings.csv file found inside the GitHub repository for this book. Also, make sure your SparkSession is initialized.

All the code in this recipe can be executed in Jupyter Notebook cells or a PySpark shell.

How to do it…

Here are the steps to perform this recipe:

1. Importing Spark data types...