Question

An SRE team supports a payment API with a well-defined availability SLO. Over the past quarter, on-call engineers report severe alert fatigue: they receive roughly 40 pages per week, but fewer than 5 require any human action. The remaining pages either auto-resolve within a minute or point to conditions that do not threaten the SLO. Leadership wants to improve on-call health while protecting reliability. Which action best addresses the root problem?

Accepted Answer

Reconfigure alerting so that pages fire on symptom-based SLO burn-rate conditions that predict user-facing impact, and route non-actionable notifications to tickets or dashboards instead of pages.

Answer

Increase the on-call rotation size so each engineer receives fewer pages per shift, keeping all existing alert rules unchanged.

Answer

Raise all alert thresholds by 50% uniformly so fewer alerts cross the firing boundary during normal operation.

Answer

Suppress all alerts that auto-resolve within one minute by adding a global 90-second delay before any page is delivered.