You're reading from Data Engineering with Apache Spark, Delta Lake, and Lakehouse Create scalable pipelines that ingest, curate, and aggregate complex data in a timely and secure way

Product type Paperback

Published in Oct 2021

Publisher Packt

ISBN-13 9781801077743

Length 480 pages

Edition 1st Edition

Languages

Python

Tools

Apache Spark

Concepts

Data Engineering

Author (1):

Manoj Kukreja

Preface

1. Section 1: Modern Data Engineering and Tools

2. Chapter 1: The Story of Data Engineering and Analytics FREE CHAPTER

3. Chapter 2: Discovering Storage and Compute Data Lakes

4. Chapter 3: Data Engineering on Microsoft Azure

5. Section 2: Data Pipelines and Stages of Data Engineering

6. Chapter 4: Understanding Data Pipelines

7. Chapter 5: Data Collection Stage – The Bronze Layer

8. Chapter 6: Understanding Delta Lake

9. Chapter 7: Data Curation Stage – The Silver Layer

10. Chapter 8: Data Aggregation Stage – The Gold Layer

11. Section 3: Data Engineering Challenges and Effective Deployment Strategies

12. Chapter 9: Deploying and Monitoring Pipelines in Production

13. Chapter 10: Solving Data Engineering Challenges

14. Chapter 11: Infrastructure Provisioning

15. Chapter 12: Continuous Integration and Deployment (CI/CD) of Data Pipelines

16. Other Books You May Enjoy

Other Books You May Enjoy

If you enjoyed this book, you may be interested in these other books by Packt:

Data Engineering with Python

Paul Crickard

ISBN: 9781839214189

Understand how data engineering supports data science workflows
Discover how to extract data from files and databases and then clean, transform, and enrich it
Configure processors for handling different file formats as well as both relational and NoSQL databases
Find out how to implement a data pipeline and dashboard to visualize results
Use staging and validation to check data before landing in the warehouse
Build real-time pipelines with staging areas that perform validation and handle failures
Get to grips with deploying pipelines in the production environment

Azure Data Engineering Cookbook

Ahmad Osama

ISBN: 9781800206557