Understanding why monolithic data platforms fail

If we look at the evolution of data management over the last 40 years, we’ll see a story of incredible technological revolutions and just as many project failures. At the beginning of this chapter, we mentioned that the main reason for these failures is the complexity generated by data management platforms, and this complexity grows approximately quadratically with the size of the platform. Therefore, these are not typical project failures as we are accustomed to understanding them. Data platforms rarely fail before their launch, never making it into production. Instead, they often experience failures related to their ability to evolve and survive over time. Platforms don’t fail immediately but over time, as they struggle to deliver the expected value in proportion to the constantly increasing maintenance costs they generate.

Like a Jenga tower becoming increasingly unstable as more pieces are added until it collapses...