Question

A financial services company runs a customer-facing chatbot on Amazon Bedrock using an on-demand foundation model. Traffic has grown steadily, and during peak hours the application now receives frequent ThrottlingExceptions because it exceeds the account's on-demand requests-per-minute service quota. The company has forecasted a predictable, sustained volume for the next 12 months and wants a stable, guaranteed level of throughput without changing model behavior or application code, while minimizing the risk of continued throttling. Which approach best meets these requirements?

Accepted Answer

Purchase Provisioned Throughput with a 1-month or longer commitment term for the model, provisioning enough model units to cover the forecasted sustained peak, and route production traffic to the provisioned model.

Answer

Submit a Service Quotas increase request for the on-demand requests-per-minute quota and continue using the on-demand model exclusively.

Answer

Enable prompt caching on the on-demand model so repeated system context is not re-billed, which will eliminate the ThrottlingExceptions during peak hours.

Answer

Migrate all peak-hour requests to Bedrock batch inference jobs to bypass the on-demand throughput quota.