Writing and Distributing Custom Providers
As you continue your journey with Airflow, you will no doubt encounter times when the providers that are supported by the core airflow community do not suit your needs. Initially, you might extend functionality with a single function, in a localized Python module, which you execute via the PythonOperator. You might even go so far as to extend a base operator class within that same module and use it directly to instantiate tasks.
After a while, someone in another group at your company or maybe just a fellow developer on the internet may describe a problem where your solution is the perfect fit. The question is, How do you get them that code in the best way possible?.
In this chapter, we’re going to show you how to build and distribute Airflow providers so that others can use it within their Airflow deployments. We’re going to use an intentionally silly problem so that our solution can stay simple.
This chapter covers...
