You're reading from Machine Learning Engineering with MLflow Manage the end-to-end machine learning life cycle with MLflow

Product type Paperback

Published in Aug 2021

Publisher Packt

ISBN-13 9781800560796

Length 248 pages

Edition 1st Edition

Tools

Maven

Concepts

Machine Learning

Author (1):

Natu Lauchande

View More author details

Table of Contents (18) Chapters

Preface

1. Section 1: Problem Framing and Introductions

2. Chapter 1: Introducing MLflow FREE CHAPTER

3. Chapter 2: Your Machine Learning Project

4. Section 2: Model Development and Experimentation

5. Chapter 3: Your Data Science Workbench

6. Chapter 4: Experiment Management in MLflow

7. Chapter 5: Managing Models with MLflow

8. Section 3: Machine Learning in Production

9. Chapter 6: Introducing ML Systems Architecture

10. Chapter 7: Data and Feature Management

11. Chapter 8: Training Models with MLflow

12. Chapter 9: Deployment and Inference with MLflow

13. Section 4: Advanced Topics

14. Chapter 10: Scaling Up Your Machine Learning Workflow

15. Chapter 11: Performance Monitoring

16. Chapter 12: Advanced Topics with MLflow

17. Other Books You May Enjoy

Monitoring data drift and model performance

In this section, we will run through an example that you can follow in the notebook available in the GitHub repository (at https://github.com/PacktPublishing/Machine-Learning-Engineering-with-MLflow/tree/master/Chapter11/model_performance_drifts) of the code of the package. We will run through the process of calculating different types of drift and exploring its integration with MLflow.

One emergent open source tool in the space of monitoring model performance is called Evidently (https://evidentlyai.com/). Evidently aids us in analyzing ML models during the production and validation phases. It generates handy reports integrated with pandas, JSON, and CSV. It allows us to monitor multiple drifts in ML models and their performance. The GitHub repository for Evidently is available at https://github.com/evidentlyai/evidently/.

In this section, we will explore the combination of Evidently with MLflow, in order to monitor data drift and...