Multi-Tenancy in Airflow
As your organization increases its adoption of Airflow, it is likely that more people and teams are going to want to use it. This increased usage means that there will be an increased load on what is a fundamentally shared architecture, so figuring out how to safely support different groups and individuals with different needs effectively is a critical step in your maturation journey.
In this chapter, we’ll discuss options and configurations to isolate workloads and share infrastructure in Airflow. Multi-tenancy can mean different things to different people, and we should start by plainly stating that, by many definitions, Airflow is not a multi-tenant application. Many of the patterns and practices presented in this chapter will help you provide for a shared environment but will not necessarily provide high levels of isolation and security.
In this chapter, we’ll cover the following areas:
- When to (and not to) choose multi-tenancy...
