Summary
In this chapter, we spent time working on creating our first DAG. We learned about creating an API key, how DAGs are constructed, and how operators work in relation to DAGs. When creating DAGs, it is important to remember that the way the tasks are broken up and created is up to the DAG author and maintainers of the code. It is much easier to troubleshoot multiple small tasks than large monolithic tasks. In addition, a consideration is to always work toward an architecture that meets both your and your user’s needs from the pipeline.
In the next chapter, we will expand on the notification task and show ways to utilize notifications in your working teams with Slack and email notifications.
