Question

A company has deployed a RAG-based customer support assistant using a foundation model and a vector store of product documentation. Users complain that the assistant sometimes gives accurate-sounding but incorrect answers. An engineer suspects the retrieval step is returning irrelevant document chunks. Which evaluation approach BEST helps confirm whether the retrieval component is the source of the problem?

Accepted Answer

Measure the relevance of the retrieved document chunks against the user query (retrieval precision/recall) before the model generates its answer

Answer

Increase the model's temperature parameter to encourage more diverse responses

Answer

Calculate the token throughput and average latency of the endpoint under load

Answer

Fine-tune the foundation model on the full product documentation corpus