Question

A financial services company runs a customer-facing chatbot on Amazon Bedrock using a foundation model. During business hours (9 AM to 6 PM), traffic is steady and predictable at roughly 40 requests per second with strict sub-second latency requirements. Outside business hours, traffic drops to near zero. The team currently uses on-demand inference but frequently hits throttling during peak hours, and cost reviews show they are paying a premium for token volume. They want to guarantee consistent throughput and latency during business hours while minimizing waste during off-hours. Which approach best meets these requirements?

Accepted Answer

Purchase No-commitment (hourly) Provisioned Throughput with the required number of model units, provisioning it at the start of business hours and deleting it at the end of each day.

Answer

Purchase a 6-month Provisioned Throughput commitment sized for peak load and keep it active 24/7 to guarantee capacity.

Answer

Remain on on-demand inference and implement exponential backoff with jitter to absorb the throttling during peak hours.

Answer

Enable a latency-optimized inference profile on on-demand mode and request a service quota increase to raise the requests-per-second limit.