Question

A financial services company runs a customer-facing chatbot on a foundation model in Amazon Bedrock. During peak trading hours, they experience unpredictable traffic spikes, and users report intermittent throttling errors and inconsistent response times. The application requires guaranteed, consistent inference capacity during these predictable high-demand windows. Which approach BEST addresses this requirement?

Accepted Answer

Purchase Provisioned Throughput for the model to reserve dedicated inference capacity with consistent performance

Answer

Continue using on-demand inference and add client-side retry logic with exponential backoff

Answer

Switch to a smaller foundation model to reduce per-request latency during peak hours

Answer

Fine-tune the model on historical chat logs to improve its throughput capacity