What is observability?
A quick Google search will give you definitions of observability in many forms from a variety of authors, vendors, and organizations. Since this book assumes you have a fair understanding of observability, we are not going to repeat a detailed definition here again. However, we will try to explain a few key concepts that are required for this book. In short, observability is not a tool, not a technology, not a strategy but a concept or a capability that will force you to think about how you are going to gain insights into the health of your application and services, at a conceptual stage of application development itself. It’s a combination of robust architecture and development practices, streamlining existing data management tools, and adopting and standardizing processes that will aid the former.
In simplistic terms, many people call observability next-generation monitoring or supercharged monitoring. But it’s fundamentally different in many ways. For starters, monitoring is fully dependent on a set of tools to generate the information required for operating a healthy application, while a highly observable system will generate the data that points toward existing problems or potential problems. For this to be achieved, the system developers and architects have to build observability capabilities into the product as a core function of the application itself, thereby reducing the dependency on external systems or tools to monitor. This is an ideal scenario; however, in reality, for observability, we have to depend on external applications to analyze the state of health of your applications and services. When observability is built within the application, it can reveal a lot more information about it, and as a result the dependencies on external systems or tools can be reduced significantly, as well as the cost.
Observability does not replace any application’s existing monitoring tools, but it standardizes and amalgamates the capabilities of Application Performance Monitoring (APM), log and metrics management tools, and the data that’s generated from applications, and effectively uses the distributed tracing methodology to achieve observability.
The Holy Grail of automation is the ability of the applications or systems to find out their issues and problems and self-heal before the users are impacted. Hence, observability can be considered a stepping stone for self-healing applications.
Throughout this book, we will use an example of a fictional company called MK Tea that supplies varieties of tea across the globe. They collect tea leaves from various locations, get them trucked to their plants, sort the tea as per flavors, quality, grade, package, and ship them all over the world. This entire process has many moving parts – each location where tea grows has different soil, moisture, and altitude characteristics; tea leaves, once collected, are packed and trucked to the plant by a trucking company; tea leaf sorting happens at the plant, which is an important, tedious, and manual process; tea leaves are crushed into powders of different grain sizes or dried and retained as leaves, packaged by machines, and shipped off to suppliers all over the world based on the demand for various flavors. We will see how observability can help MK Tea manage its overwhelming process, which involves human labor, skilled technicians, and fully automated machines.
You can use this example as inspiration to plan for observability in your organization.
