Question

A retail company has a production Copilot Studio agent that answers product return questions. Customer satisfaction (CSAT) scores have plateaued, and the architect suspects the agent's instructions could be improved. Before rolling a revised prompt to all users, the team wants empirical evidence that the new instruction set actually improves resolution quality without regressing other conversations. Which tuning approach best supports this goal?

Accepted Answer

Run a controlled A/B experiment that routes a portion of live traffic to the revised prompt variant, then compare resolution and CSAT metrics between the control and treatment groups

Answer

Deploy the revised instructions to all users immediately and compare CSAT month-over-month in Copilot Studio analytics

Answer

Ask a small group of internal reviewers to manually rate ten sample conversations against the old and new instructions

Answer

Increase the model temperature on the existing prompt to encourage more varied and helpful responses