Question

A retail company launched a gen AI assistant that answers customer questions about return policies. After launch, the product team wants to systematically improve output quality over time. They already have a rubric and a golden dataset. What ongoing practice would MOST effectively drive continuous improvement of the assistant's real-world responses?

Accepted Answer

Collect and review a sample of real user interactions on a regular cadence, label problem cases, and feed those insights back into prompt and grounding improvements

Answer

Increase the model's temperature setting so responses feel more natural and varied to customers

Answer

Switch to the largest available foundation model, since bigger models eliminate the need for further evaluation

Answer

Freeze the prompt and configuration after launch to keep the assistant's behavior perfectly consistent