Understanding the data life cycle
Understanding the data life cycle is the first principle in becoming a data engineer. If you’ve worked with data, you must know that data doesn’t stay in one place; it moves from one storage to another, from one database to another database. Understanding the data life cycle means you need to be able to answer these sorts of questions if you want to display information to your end user:
- Who will consume the data?
- What data sources should I use?
- Where should I store the data?
- When should the data arrive?
- Why does the data need to be stored in this place?
- How should the data be processed?
To answer all those questions, we’ll start by looking back a little bit at the history of data technologies.
Understanding the need for a data warehouse
Data warehouse is not a new concept; I believe you’ve at least heard of it. In fact, this terminology is no longer appealing. In my experience, no...
