Tracing with OpenTelemetry

OpenTelemetry (OTel) is a CNCF project that provides a standard way to send application trace information. Traces become important as you begin to build a distributed system as you need to be able to trace a request and response through multiple systems. OTel provides a vendor-agnostic instrumentation library that can be used to forward trace data to a backend to visualize or analyze the data.

In this book, we will use ADOT as the trace forwarder and AWS X-Ray to analyze and visualize the trace data.

Modifying our ADOT configuration

As X-Ray is an AWS service, we first need to modify the service account permissions to allow it to send traces to X-Ray, as currently it only has AMP permissions. If we view the current ServiceAccount, we can get the IAM role, arn, as shown next:

$ kubectl get sa amp-iamproxy-ingest-role -n prometheus -o json
{
    "apiVersion": "v1",
    "kind"...