Question

A team is deploying a foundation model to answer employee questions about company HR policies. Before release, they must objectively measure how often the model's answers are factually correct against a set of questions with known, verified answers. Which evaluation approach best meets this requirement?

Accepted Answer

Create a benchmark dataset of question-and-answer pairs with verified ground-truth answers, then measure the model's accuracy against it

Answer

Lower the temperature parameter to 0 so the model produces more deterministic responses

Answer

Ask a panel of employees to rate whether each response sounds professional and polite

Answer

Increase the context window size so the model can process longer policy documents