Question

A Generative AI Engineer runs Mosaic AI Agent Evaluation on a deployed RAG chatbot. The overall answer-correctness score is low. The evaluation report shows that the retrieval metrics (context precision and context recall) are strong, but the answer-groundedness metric is poor. Which conclusion best explains this pattern, and what should the engineer prioritize fixing?

Accepted Answer

The correct context is being retrieved, but the LLM is generating claims not supported by that context; the engineer should focus on the generation prompt and model, not the retriever.

Answer

The retriever is pulling irrelevant documents; the engineer should tune the vector search index and re-embed the corpus.

Answer

The golden dataset labels are wrong; the engineer should discard the evaluation results and rely only on production inference tables.

Answer

The chunk size is too small; the engineer should increase chunk size to improve both retrieval and groundedness simultaneously.