Introduction

Microservices add complexity to an architecture. With more moving parts in a system, monitoring and observing the behavior of the system becomes more important and more challenging. In a microservice architecture, failure conditions impacting one service can cascade in unexpected ways, impacting the system as a whole. A faulty switch somewhere in a datacenter may be causing unusually high latency for a service, perhaps resulting in intermittent timeouts in requests originating from the API Gateway, which may result in unexpected user impact, which results in an alert being fired. This kind of scenario is not uncommon in a microservice architecture and requires forethought so that engineers can easily determine the nature of customer-impacting incidents. Distributed systems are bound to experience certain failures and special consideration must be taken to build observability...