From Data as a Byproduct to Data as a Product

In this book, we will explore how to transition from managing data merely as a byproduct that supports applications to managing data as a product in its own right. Before tackling the various aspects that contribute to this paradigm shift, it’s crucial to understand why managing data as a product is important and how this practice enables us to surpass the limits of today’s data platforms.

In this chapter, we will explore the history of monolithic data platforms, which have characterized the evolution of data management over the last 30 years. We will seek to understand the common problems that make them incapable of sustainably managing the accidental complexity they generate as they grow. Finally, we will see why addressing the fundamental issues, instead of merely treating surface-level symptoms, requires more than just technological innovations. It calls for a paradigm shift that leads us toward more sustainable socio...