Question

Your team has deployed a customer-support RAG application to production using Azure AI Foundry. During pre-deployment you already ran batch evaluations against a curated test dataset and the app passed. However, leadership now wants ongoing visibility into groundedness and relevance degradation on real user traffic as it happens, with the results surfaced automatically without manually assembling new test datasets. Which approach should you implement?

Accepted Answer

Configure continuous (online) evaluation in Azure AI Foundry, wiring evaluators such as groundedness and relevance to sampled production traces so metrics are computed and emitted automatically.

Answer

Re-run the same offline batch evaluation on the curated test dataset every night via a scheduled pipeline and review the aggregated report each morning.

Answer

Enable Azure Content Safety filters on the deployment so that low-groundedness responses are automatically blocked before reaching users.

Answer

Export raw request/response logs to a storage account and have analysts periodically read samples to judge answer quality by hand.