Core Airflow Concepts
Apache Airflow is built upon the main concepts that will be discussed in this chapter. At the heart of Airflow are the main concepts that streamline the process of defining, executing, and monitoring tasks. These concepts include tasks, task groups, and triggers. Each of these constitutes directed acyclic graphs (DAGs) and allows you to take advantage of Airflow. Understanding each of these building blocks is essential for leveraging Airflow’s full potential at scale and ensuring that workflows are automated and optimized.
In addition to the basics of DAGs, this chapter will introduce the concept of a command-line interface to run Apache Airflow locally on a computer or a virtual machine. The process is easy to follow, and multiple tools have been created to initiate this process. The same tools that we will use to begin creating DAGs and setting up Airflow can be used in more complex situations as well, so these tools are industry standards.
In this...
