Moving Beyond the Basics
In the previous chapters, we discussed the basic tenets of data engineering, our opinionated approach to the Pragmatic Data Platform (PDP), and we used basic and advanced dbt functionalities to implement it in its basic form.
In this chapter, you will review the best practices to apply modularity in your pipelines to simplify their evolution and maintenance.
Next you will learn how to manage the identity of your entities as it is central to store changes to them and to apply master data management to combine data from different systems.
We will also use macros, the most powerful dbt functionality, to implement the first pattern to store and retrieve the changes in our data according to our discussion of identity management. This allows all developers to use the best practices that senior colleagues have developed.
In this chapter, you will learn about the following topics:
- Building for modularity
- Managing identity
- Master data management...
