Question

Your team is configuring an automated evaluation run in Azure AI Foundry for a RAG-based customer support agent. You have a JSONL test dataset whose columns are named 'user_question', 'retrieved_docs', and 'model_reply'. You want to run the built-in groundedness and relevance evaluators, but the evaluation run fails because the evaluators cannot find the expected inputs. What should you do to resolve this without renaming the dataset columns?

Accepted Answer

Configure data mapping to bind the evaluator inputs (query, context, response) to the dataset columns 'user_question', 'retrieved_docs', and 'model_reply'.

Answer

Convert the JSONL file to CSV format because built-in evaluators only accept tabular CSV test datasets.

Answer

Add a custom Python evaluator that reads the raw columns directly, since built-in groundedness and relevance evaluators cannot be reused with custom schemas.

Answer

Enable the content safety evaluators, which automatically infer input columns from any dataset regardless of column names.