When to choose multi-tenancy
Multi-tenancy can mean many things to many people and for the duration of this chapter we’re going to use the working definition of multiple individuals with different needs utilizing a shared Airflow instance. This wish usually comes from a need to ensure that the organization is getting the most out of its Airflow deployment, ensuring that it is being used to its utmost potential.
The choice to run Airflow in a multi-tenant manner should not be made lightly, and we have included this chapter partially as a warning that it should be avoided. With the rise of Kubernetes, community-supported Helm charts, and paid Airflow services, it is generally preferable to operate an Airflow per organizational unit (team, business, or security group) than attempt to manage large numbers of conflicting needs within a single deployment.
If you find yourself in a situation where you cannot avoid sharing a deployment, we’ll present some technical guidelines...
