Summary
In this chapter, we learned about the data pipeline life cycle. Hopefully, I was able to highlight the fact that the discovery sessions and pipeline design are more important than the development itself. Similarly, it is very important to keep the stakeholders in the loop with everything that is happening. It is important that data engineers conduct regular checkpoints, demonstrations, and code review sessions with the stakeholders.
It is time to put the cloud services that we learned about during Chapter 3, Data Engineering on Microsoft Azure, to work. In the next chapter, we will start to use the Azure services and highlight the various stages of data engineering.
