Question

A financial analytics team runs a high-volume document summarization service on Amazon Bedrock using an on-demand foundation model. During peak business hours, their application experiences bursts of requests and begins receiving intermittent ThrottlingException errors. The team wants to maximize successful request completion during these bursts without immediately purchasing Provisioned Throughput, and they want to avoid overwhelming the service with synchronized retries. Which approach BEST addresses the throttling while improving overall throughput resilience?

Accepted Answer

Implement client-side retries using exponential backoff with jitter for ThrottlingException responses, and distribute traffic across a cross-region inference profile to spread load.

Answer

Configure a fixed 1-second retry interval on every failed request so all clients retry at a predictable, consistent cadence.

Answer

Increase the max_tokens parameter on each request so fewer total requests are needed to summarize each document.

Answer

Disable automatic retries entirely and immediately fail requests to the user so they can manually resubmit during quieter periods.