Setting up StatsD for monitoring

As introduced in Chapter 10, StatsD is an open source daemon that gathers and aggregates metrics about application behaviors. Due to its flexibility and lightweight, StatsD is used on several monitoring and observability tools, such as Grafana, Prometheus, and ElasticSearch, to visualize and analyze the collected metrics.

In this recipe, we will configure StatsD using a Docker image as the first step in building a monitoring pipeline. Here, StatsD will collect and aggregate Airflow information and make it available to Prometheus, our monitoring database, in the Setting up Prometheus for storing metrics recipe.

Getting ready

Refer to the Technical requirements section for this recipe since we will handle it with the same technology.

How to do it…

Here are the steps to perform this recipe:

1. Let’s start by defining our Docker configurations for StatsD. These lines will be added under the services section inside the docker...