Basics of Airflow and DAG Authoring
In Chapter 3, we went through the basics of setting up Airflow on your local machine and analyzing a basic ETL example. In this chapter, we will take it a large step further with a real-world example of extracting data from an API on a schedule. You can expect to use the knowledge from the previous chapters to tackle this problem, and we will guide you along the way.
The best way to become familiar with Airflow and understand the basics is to get “hands-on” with the process. As we will be providing complex examples in the following chapters and identifying areas of improvement in the cloud for cost and efficiency savings, it is important that these early examples lead up to those in appropriate levels of complexity.
In this chapter, we are going to cover the following main topics:
- Authoring an advanced DAG example
- Learning about operators
- Presenting findings and reviewing the information
