Question

A retail company runs a customer-support summarization service on Amazon Bedrock using an on-demand Anthropic Claude model. During weekday business hours, traffic is steady and well within account limits, but a scheduled 30-minute nightly job processes a large backlog and consistently triggers ThrottlingException errors as it exceeds the account's requests-per-minute quota. The nightly job is not latency-sensitive and can tolerate delays, but it must complete reliably each night. The team wants the most cost-effective way to eliminate the throttling without over-provisioning capacity that would sit idle during the day. What should they do?

Accepted Answer

Refactor the nightly job to use Amazon Bedrock batch inference, submitting the backlog as a batch job instead of real-time InvokeModel calls.

Answer

Purchase Provisioned Throughput with enough model units to cover the nightly peak so the reserved capacity absorbs the burst.

Answer

Request a permanent service quota increase for on-demand requests-per-minute to a level that covers the nightly peak.

Answer

Add client-side exponential backoff with jitter to the nightly job and keep using on-demand invocations at the current quota.