Question

A financial services company runs a customer-facing chatbot on Amazon Bedrock using Anthropic Claude. Traffic is highly variable: near zero overnight, moderate during business hours, and short unpredictable spikes when market news breaks. During spikes the team occasionally hits on-demand throughput quotas, causing throttling errors. Finance wants to minimize monthly cost while eliminating throttling during spikes. Which approach best meets these requirements?

Accepted Answer

Keep using on-demand inference and implement exponential backoff with retries in the application to handle throttling gracefully during spikes

Answer

Purchase a large Provisioned Throughput commitment sized for peak spike traffic with a 6-month commitment term

Answer

Switch entirely to Provisioned Throughput with no commitment (hourly) sized to the average business-hours load

Answer

Migrate the workload to a smaller, cheaper model that has higher default on-demand quotas regardless of output quality