Question

A DevOps team runs an automated incident-response workflow: an EventBridge rule matches CloudTrail management events for security-group ingress changes and invokes a Lambda function that reverts non-compliant rules. During a recent audit, the team found that some non-compliant rule additions were reverted several minutes late, and a few were never reverted at all. CloudWatch Logs show the Lambda function was invoked far fewer times than the number of AuthorizeSecurityGroupIngress API calls recorded in CloudTrail. The EventBridge rule pattern and Lambda IAM permissions are confirmed correct. What is the MOST likely root cause of the missed and delayed remediations?

Accepted Answer

CloudTrail management events are delivered to the default event bus on a best-effort basis and can be delayed or, in rare cases, not delivered, so relying solely on CloudTrail events for time-sensitive remediation is unreliable.

Answer

The EventBridge rule is matching against a CloudTrail trail's S3 delivery events, which are batched and delivered with delay, so many API calls never generate a matching event.

Answer

The Lambda function's reserved concurrency is set too low, causing EventBridge to drop events once concurrency is exhausted rather than retrying them.

Answer

EventBridge deduplicates identical event payloads within a 5-minute window, so repeated AuthorizeSecurityGroupIngress calls with similar parameters are collapsed into a single invocation.