Question

Your team maintains a chat prompt for a customer-support assistant in Azure AI Foundry. Before merging a proposed prompt change into the main branch, you must objectively measure whether the new prompt produces better answers across a representative set of 500 historical support questions with known ideal answers. You want a repeatable, quantitative comparison rather than manually inspecting individual outputs. Which approach should you use?

Accepted Answer

Run a batch evaluation in prompt flow using the 500-question dataset and an evaluation flow that scores each response against the ground-truth answers, then compare aggregate metrics between the two prompt versions.

Answer

Deploy both prompt versions to separate serverless API endpoints and route production traffic to each, then compare content safety filter logs.

Answer

Manually enter several sample questions into the prompt flow chat interface and visually inspect the differences between the two prompt versions.

Answer

Increase the provisioned throughput units for the deployment so both prompt versions can be tested at higher volume before merging.