Alerts and incidents – the finer details

We introduced alerts and incidents in the previous section. They are like wake-up calls about anything that deserves attention in an IT environment when something needs to be notified or something is not behaving as expected. In this section, we will answer the following fundamental questions:

• At what point should an alert be set up?
• What should be the frequency of the alert?
• How should the alert behave for the first and consecutive events?
• Who are the consumers of alerts?

That is a lot to understand, and as simple as it may sound, it needs a team of people with varied skills to identify alert touchpoints, develop them, respond to them, and keep them relevant. An observability team needs members with all these skills, and for those members to work closely with application SMEs to create quality alerts.

Let’s start answering the questions one by one.

At what point should an alert be set up?

Any...