Question

A financial services firm deploys an internal LLM chatbot to answer employee questions about company policies. Users complain the model confidently gives incorrect answers about policies that changed after the model was trained. The team wants to ground responses in the current, authoritative policy documents without retraining the model. Which approach best addresses this?

Accepted Answer

Implement retrieval-augmented generation (RAG) that fetches relevant policy passages from a vector database and includes them in the prompt context

Answer

Increase the model's temperature setting so it generates more varied responses

Answer

Reduce the context window size to force the model to give shorter answers

Answer

Deploy a larger version of the same base model with more parameters