Question

A Business Solutions Architect is preparing to release a new version of a Copilot Studio agent that handles order-status inquiries integrated with Dynamics 365 Supply Chain. Leadership requires proof that the new version performs at least as well as the current one on real production questions before it starts responding to any customers. They want to compare the new version's generated responses against actual live traffic without exposing customers to the new version's answers or affecting the current customer experience. Which testing approach best meets this requirement?

Accepted Answer

Deploy the new version in shadow (mirrored) mode so it processes copies of live production requests silently while the current version continues to serve all customer responses, then compare the shadow outputs offline.

Answer

Run a canary deployment that routes 10% of live customer traffic to the new version and monitors error rates before expanding.

Answer

Execute a synthetic regression test suite of scripted utterances in a non-production environment and promote once all cases pass.

Answer

Enable A/B testing that randomly splits customers between the two versions and compares satisfaction scores over two weeks.