Question

A GenAI team runs a customer support chatbot on Amazon Bedrock. Finance reports that daily inference costs occasionally spike 5x with no corresponding increase in user traffic. The team suspects certain conversations are consuming abnormally large token counts, but current monitoring only shows aggregate invocation counts and total cost. They need to identify which specific requests drive the cost spikes and correlate them with prompt content so they can add controls. What is the MOST effective approach to gain this visibility?

Accepted Answer

Enable Bedrock model invocation logging to deliver full request/response payloads (including input and output token counts) to CloudWatch Logs and S3, then query with CloudWatch Logs Insights to find high-token invocations.

Answer

Enable CloudTrail data events for Bedrock InvokeModel calls and analyze the event history to determine per-request token consumption.

Answer

Add a CloudWatch metric alarm on the aggregate InputTokenCount metric and increase the sampling frequency to one-second resolution.

Answer

Use AWS Cost Explorer grouped by usage type with hourly granularity to isolate the individual conversations causing the token spikes.