When people talk about observability, it’s usually in the context of obtaining data (metrics, logs, and traces) for the purpose of resolving incidents. But what if that data could be used for more than emergency situations? And what if expanding our understanding of what observability is can help us better resolve incidents – and maybe even prevent them in the first place?
Observability is not just for the acute moments of resolving emergency situations. Instead, it’s actually for three different things: 1.) defining the health of a system, 2.) addressing issues, and 3.) learning so we can improve. Read on for more details.
#1: Understanding the Health of a System
The first and most foundational use for observability is to understand the health of a system, – especially as that health occurs in real-time. Monitoring the state of your system is important because it creates a baseline for what healthy looks like, so you can quickly and clearly spot when it is unhealthy.
Simply put, observability is how we know when our systems need attention. Preparing yourself with this capability through tracking metrics such as the four golden signals sets you up to find out about issues quickly.
However, observability for system health isn’t just about alerting you to issues. Continuous monitoring of system health can also inform proactive performance improvements, and help you prevent incidents from occurring..
So far, observability allows us to identify deviations from system health. But that information in itself isn’t enough to address why the issue is unhealthy in the first place. Lucky for you, observability also allows us to identify these “root causes” as well.
#2: Resolving Incidents
Observability not only allows us to understand the baseline health of a system, it also allows us to identify where things are going wrong. Some of the data that helps us make these assessments exist in two main groups:
- Metrics which are a great way to understand system health.
- Logs and traces which are ideal tools for triaging and fixing system problems.
These two types of data provide us with the details we need about the issue, such as where it’s occurring, to what extent it’s happening, and who it’s affecting.
Having a high-level of detail is immensely useful when trying to resolve an issue. However, if this data isn’t properly organized and understood, it can simply make finding the “needle in the haystack” that is your issue even harder to find.
#3: Learning and improving
So when setting up observability to support incident response, it’s best to keep the complexity of data – and what that data can do for you – in mind. Some of this complexity also includes how different services connect, and how transactions go across them, including with third-party cloud dependencies.
Finally, observability is a key driver to help us learn and improve. You are hopefully already using your learnings from incidents to improve your incident response process, and observability data can be a useful tool for learning and improving as well.
Whether it’s using the software signals to update your definition of health, to find a way to be alerted sooner, or to change your process of incident response, don’t forget that observability is there to help you learn and improve both your software systems, and your human systems.
Breaking Out of the Pillars
The sooner we break away from defining observability through the lens of the different forms data can come in, the sooner we’ll be able to realize the promise of observability. Thinking about observability through the lens of how the data can empower you in practical ways can lead not only to better reliability for your systems, but also to a more efficient, and joyful experience with your observability practice.
