Question

A Business Solutions Architect is finalizing testing for a custom document-classification model built in Microsoft Foundry that routes incoming supplier claims into 12 categories in Dynamics 365 Supply Chain. Misrouting a high-value warranty claim to the wrong queue causes costly delays, while occasionally routing a routine claim for manual review is acceptable. The team must define validation criteria before promoting the model to production. Which validation approach best fits the business risk profile?

Accepted Answer

Prioritize high recall for the high-value warranty categories against a held-out labeled test set, accepting lower precision that sends borderline cases to manual review.

Answer

Set a single overall accuracy threshold of 95% across all categories and promote once it is reached in aggregate testing.

Answer

Maximize precision across all 12 categories equally so that every automated routing decision is correct, ignoring recall entirely.

Answer

Validate the model only against the same data used for training to confirm it has learned the categories correctly.