Best practices and approaches

In this chapter, we have provided an overview of several popular cloud infrastructures. Let’s now discuss some of the best practices that should be considered when implementing observability on any application or system:

• Performance: The process of retrieving telemetry data can potentially incur a performance overhead. For example, with a remote Grafana data source, the telemetry data is fetched at query time over a great distance. This can introduce latency when compared to data stored closer to the Grafana query engine using one of the Grafana Cloud data sources, such as Loki, Mimir, and Tempo. Where performance is important and there is an option to ship telemetry into Grafana, that could be the best choice. Alternatively, several data sources have caching options to improve query speed; improvements in query speed can also be made using specific configurations. Take the time to understand your data and ensure you are using it in an optimal...