Question

Your team is preparing to promote a customer-support generative AI application from staging to production in Azure AI Foundry. Compliance requires that, before each release, the app be tested against a curated dataset of hundreds of representative and adversarial prompts to produce quantitative scores for groundedness, relevance, and content harm categories such as violence and self-harm. The results must be reviewable as a report and comparable across releases. Which approach in Azure AI Foundry best meets this requirement?

Accepted Answer

Run a batch evaluation using Foundry evaluators (including AI-assisted quality and risk/safety evaluators) against the curated test dataset and review the aggregated evaluation run results.

Answer

Enable Prompt Shields on the deployment so jailbreak attempts are blocked at inference time for each incoming request.

Answer

Configure Azure Monitor alerts on the production endpoint to notify the team when latency or token usage crosses defined thresholds.

Answer

Attach a content safety filter to the model deployment and rely on its per-request blocking to validate the release.