Creating custom dashboards to monitor Data Lake services
CloudWatch dashboards allow you to create interactive visualizations of your data, giving you a consolidated view of the health and status of your AWS assets. CloudWatch provides several pre-built dashboards for various services, but you can also create custom dashboards to meet your specific needs.
In the daily operation, you should have a high-level dashboard built from CloudWatch log groups metrics to help you understand the general performance and perform drill-down on services if needed, such as creating a high-level dashboard on ETL performance and then drilling down to Glue job run monitoring if required for further investigation.
Getting ready
To proceed with this recipe, you need a CloudWatch log group for the services for which you want to build a dashboard.
How to do it…
- Open the AWS console and select the CloudWatch service, which will present you with a screen that is similar to the following...
