Question

A Generative AI Engineer is building a customer-support chatbot on Databricks that answers questions from a product documentation knowledge base. Users frequently ask follow-up questions such as 'What about the enterprise tier?' that only make sense in the context of earlier turns in the conversation. Testing shows that when these follow-up questions are sent directly to the vector search retriever, the retrieved chunks are irrelevant because the standalone query lacks context. Which design change best addresses this problem while keeping retrieval accurate?

Accepted Answer

Add a query-rewriting step that uses the LLM and conversation history to reformulate the follow-up into a self-contained standalone query before retrieval.

Answer

Increase the number of chunks (top_k) returned by the vector search index so that more context is available to the LLM.

Answer

Concatenate the entire raw conversation history and embed it directly as the retrieval query for every turn.

Answer

Switch the chunking strategy to larger fixed-size chunks so each retrieved document contains more surrounding context.