What this book covers
Chapter 1, Getting Started with Airflow 2.0, offers a brief introduction to what Airflow is and why it is an important component of today’s data ecosystem.
Chapter 2, Core Airflow Concepts, covers core concepts, objects, and ontologies important for understanding and describing how Airflow functions.
Chapter 3, Components of Airflow, covers core operating components of Airflow, what they are responsible for, and generally how they work.
Chapter 4, Basics of Airflow and DAG Authoring, covers the basic process of writing a workflow in Airflow.
Chapter 5, Connecting to External Sources, covers how to set up and configure external sources of information/interaction with Airflow.
Chapter 6, Extending Functionality with UI Plugins, covers how to customize Airflow’s UI with a custom UI plugin, allowing for streamlined organization-specific operational needs.
Chapter 7, Writing and Distributing Custom Providers, covers the process of how to apply software engineering principles to write, test, and publish Airflow providers.
Chapter 8, Orchestrating a Machine Learning Workflow, demonstrates a relatively common use case in Airflow of orchestrating the training and delivery of a machine learning model.
Chapter 9, Using Airflow as a Driving Service, demonstrates how Airflow can be used as a driving service, allowing consumers to use an abstracted interface instead of directly writing workflows.
Chapter 10, Airflow Ops: Development and Deployment, covers and demonstrates how to customize your Airflow deployment, apply version control to your workflows, and develop a development life cycle for the delivery and maintenance of workflows.
Chapter 11, Airflow Ops Best Practices: Observation and Monitoring, covers practices for monitoring and maintaining your core Airflow deployment and your operational DAGs.
Chapter 12, Multi-Tenancy in Airflow, covers how to design, configure, and operate your Airflow deployment to achieve differing levels of multi-tenancy and compute isolation.
Chapter 13, Migrating Airflow, discusses practical guidance in planning and executing a migration of your Airflow instance from one environment to another.
