Question

A retail company runs a fleet of hundreds of EC2 instances and containerized microservices generating unstructured, plaintext application logs written to local disk. During incidents, on-call engineers must SSH into individual hosts to grep logs, which delays root-cause analysis. Leadership wants to improve observability so that engineers can query logs across all services centrally, automatically detect anomalous error-rate spikes without writing custom parsing scripts, and receive proactive alerts. The solution must minimize ongoing operational overhead. Which approach best meets these requirements?

Accepted Answer

Install the CloudWatch agent to ship structured logs to CloudWatch Logs, use CloudWatch Logs Insights for centralized querying, and configure a CloudWatch Logs metric filter with an anomaly-detection alarm to alert on error-rate spikes.

Answer

Configure each instance to stream logs to an Amazon S3 bucket via a cron job, then run scheduled Amazon Athena queries and build a custom Lambda function to parse results and publish SNS alerts on threshold breaches.

Answer

Deploy a self-managed Elasticsearch cluster on EC2 with Logstash agents on each host, build Kibana dashboards, and write custom Watcher rules for error-rate alerting.

Answer

Enable VPC Flow Logs and AWS CloudTrail across all accounts, aggregate them in a central logging account, and use Amazon Detective to identify anomalous application error patterns.