Configuring Airflow

Apache Airflow has many capabilities and a quick setup, which helps us start designing our workflows as code. Some additional configurations might be required as we progress with the workflows and into data processing. Gladly, Airflow has a dedicated file for inserting other arrangements without changing anything within its core.

In this recipe, we will learn more about the airflow.conf file, how to use it, and other valuable configurations required to execute the other recipes in this chapter. We will also cover where to find this file and how other environment variables work with this tool. Understanding these concepts in practice helps us to identify potential improvements or solve problems.

Getting ready

Before moving on to the code, ensure your Airflow runs correctly. You can do that by checking the Airflow UI at this link: http://localhost:8080.

If you are using a Docker container (as I am) to host your Airflow application, you can check its status...