Question

Your team runs a production chat application against a serverless API endpoint for a foundation model in Azure AI Foundry. During peak business hours, users report intermittent HTTP 429 responses, but the application stays fully functional during off-peak times. Traffic is highly variable and unpredictable, and management does not want to commit to a fixed monthly capacity spend. What should you do to reduce the 429 errors while keeping cost aligned to actual usage?

Accepted Answer

Request a tokens-per-minute (TPM) quota increase for the serverless deployment and implement client-side exponential backoff with retries.

Answer

Purchase provisioned throughput units (PTUs) sized for the peak traffic to guarantee capacity at all times.

Answer

Switch the deployment to managed compute with a fixed-size GPU instance pool.

Answer

Enable the content safety filter on the serverless endpoint to smooth out request bursts.