Question

A retail company wants a customer-service assistant that can answer questions using their 50,000-page internal knowledge base, which is updated weekly. The AI transformation leader is comparing two approaches: (1) sending large portions of the knowledge base into an expanded model context window with every request, or (2) implementing a retrieval-augmented generation (RAG) architecture with a search index. Which factor most strongly favors the RAG approach for this use case?

Accepted Answer

RAG retrieves only the most relevant passages per query, reducing token consumption and cost while keeping answers current as the index is refreshed.

Answer

RAG permanently embeds all knowledge base facts into the model's weights, eliminating the need to re-index when content changes.

Answer

RAG removes the need for prompt engineering because retrieved documents automatically guarantee factually correct responses.

Answer

RAG allows the company to use a smaller pretrained model that requires no grounding data of any kind.