Baselining and scheduled monitoring with SageMaker Model Monitor
In the previous recipe, we deployed the model to an inference endpoint and enabled data capture using SageMaker Model Monitor. This allows us to collect the request and response pairs when the endpoint is invoked during inference. Note that we are just scratching the surface here in terms of what we can do with SageMaker Model Monitor. Using SageMaker Model Monitor, we can also automatically monitor and detect the following issues:
- Drift in data quality
- Drift in model quality metric values
- Bias drift during prediction
- Feature attribution drift
This is important as there's a lot of things that can happen after our model gets deployed to production.
In this recipe, we will focus our efforts on detecting data quality drift. We will start by preparing a baseline, and then creating a scheduled monitoring job that processes the data captured by Model Monitor and outputs summary statistics...
