Question

A retail company runs a customer-support agent on an Azure AI Foundry standard (pay-as-you-go) model deployment. During flash sales, request volume spikes for short periods and clients begin receiving HTTP 429 responses, causing failed conversations. The team wants to reduce user-facing failures during these bursts without over-provisioning capacity for the rest of the day, and they can tolerate a small increase in latency during peaks. What should they do?

Accepted Answer

Implement client-side exponential backoff with retry on 429 responses and honor the Retry-After header returned by the service.

Answer

Migrate the deployment to a provisioned throughput unit (PTU) allocation sized for the peak sale traffic.

Answer

Increase the deployment's temperature and max token settings so responses complete faster and free up capacity.

Answer

Disable the content safety filters during sales events to lower per-request processing overhead.