Question

A Generative AI Engineer wants to run Mosaic AI Agent Evaluation on a deployed RAG chatbot using an offline evaluation dataset. The team has collected 200 representative user questions and expected reference answers, but no retrieval traces. They want to compute both answer correctness (comparing to the expected answers) and groundedness (whether responses are supported by retrieved context). Which approach correctly structures the evaluation so both metric types can be computed?

Accepted Answer

Provide a dataset with 'request' and 'expected_response' columns, and let Agent Evaluation invoke the agent so that retrieved context is captured in the trace for groundedness scoring.

Answer

Provide only a 'request' column with the 200 questions; Agent Evaluation will infer expected answers and retrieved context automatically.

Answer

Provide 'request' and 'expected_response' columns only; groundedness cannot be computed without manually pre-populating a 'retrieved_context' column for every row.

Answer

Split the work into two datasets—one with 'request' and 'expected_response' for correctness, and a separate one with only 'request' for groundedness—since the two metrics cannot share an evaluation run.