Question

A media company runs a Bedrock-powered content generation service using an on-demand foundation model. During a marketing campaign, they observe a predictable, sustained baseline of roughly 90 requests per second for 8 hours each business day, plus unpredictable short spikes that occasionally double that volume for a few minutes. On-demand throttling during peaks is causing failed generations, and finance wants to reduce the per-token cost of the steady baseline. Which approach BEST balances cost efficiency and reliability?

Accepted Answer

Purchase Provisioned Throughput with enough model units to cover the sustained baseline and rely on on-demand capacity with exponential backoff to absorb the short unpredictable spikes.

Answer

Purchase Provisioned Throughput sized for the peak (double the baseline) so that no request is ever throttled, accepting the higher committed cost.

Answer

Stay entirely on on-demand and request a large service quota increase to eliminate throttling during both baseline and spikes.

Answer

Move all baseline and spike traffic to Batch inference to eliminate throttling and lower per-token cost.