Question

You manage observability for a SecOps platform that ingests logs from 40 log sources. Leadership wants an automated way to catch a log source that has stopped sending data OR has abnormally spiked, before it affects detection coverage. Some sources are batch-based (bursty every 6 hours) and some stream continuously. Which monitoring approach best balances early gap detection with minimizing false alarms across these mixed ingestion patterns?

Accepted Answer

Compute a rolling per-source ingestion baseline over a comparable historical window and alert on statistically significant deviations (both drops and spikes) relative to each source's own expected pattern

Answer

Alert whenever any log source's events-per-second drops to zero for more than 5 minutes, applied uniformly to all 40 sources

Answer

Set a single global daily ingestion volume threshold across all 40 sources and alert if total volume falls below it

Answer

Rely on parser error rates as the primary signal, since a source that stops sending will produce parser failures