Question

A media company runs a document summarization service on Amazon Bedrock using Claude on-demand throughput in us-east-1. During peak evening hours, the service processes bursts of user requests and increasingly receives ThrottlingException errors, causing dropped summaries. The workload is latency-sensitive (users wait for results interactively) and cannot tolerate the multi-hour delay of asynchronous processing. Traffic is unpredictable and spiky, so committing to a fixed hourly rate is undesirable. The team wants to increase available throughput headroom with minimal architectural change. Which approach BEST addresses the throttling while fitting the workload's constraints?

Accepted Answer

Switch the invocations to use a Bedrock cross-region inference profile so requests are automatically distributed across multiple regions, increasing aggregate on-demand throughput capacity.

Answer

Migrate the workload to Bedrock batch inference to avoid on-demand throttling limits during peak hours.

Answer

Purchase Provisioned Throughput with a 1-month commitment sized to peak traffic to guarantee capacity.

Answer

Reduce the max_tokens parameter on each request to lower per-request token consumption and stay under the throttling limit.