Question

A financial services company runs a customer-facing chatbot on Azure AI Foundry using a GPT-4 model deployed to a serverless (pay-as-you-go) API endpoint. During peak trading hours the team observes rising latency and intermittent HTTP 429 responses, and the business now requires a guaranteed, consistent response-time SLA for a predictable high volume of requests. Cost predictability is also a stated goal. Which deployment change best meets these requirements?

Accepted Answer

Provision the model on a Provisioned Throughput Units (PTU) deployment sized to the expected peak load.

Answer

Keep the serverless endpoint and submit a quota increase request to raise the tokens-per-minute limit.

Answer

Deploy multiple serverless endpoints behind a load balancer and distribute requests round-robin.

Answer

Enable autoscale on the serverless endpoint so replicas scale out during peak trading hours.