Sending metrics to Prometheus with the OpenTelemetry collector

At the time of writing, sending data to Prometheus over OTLP is still considered an experimental feature. In fact, the functionality came out during the process of writing this book (in Prometheus version 2.47.0), so we’ll need to upgrade the version of Prometheus we’ve been running so far in order to enable support for it.

Configuring Prometheus

To support the OTLP receiver on Prometheus, we need to adjust our Helm chart to install a newer version of Prometheus and add a --enable-feature=otlp-write-receiver flag to the Prometheus process. Additionally, to avoid any complications inherent in pushing data to multiple Prometheus replicas, we’ll scale back down to a single replica.

The Helm values file we’ll use looks like the following:

grafana:
  enabled: true
  defaultDashboardsTimezone: browser
  adminUser: root
  adminPassword: m@ster1ngPr0m3th3us...