🔥 3-day streak
Microsoft Machine Learning Operations Engineer Associate58 / 144
Question 58 of 144

Your team deploys a fine-tuned open-source language model to a managed compute endpoint in Azure AI Foundry. During load testing, the endpoint sustains high GPU utilization but throughput plateaus and request latency climbs sharply as concurrent traffic increases. The model must remain always-on for an internal application with steady, predictable demand. You need to increase throughput without changing the model. What should you do first?

Reviewed for accuracy · Report an issueNext question