Question

You are building a RAG solution in Azure AI Foundry over a corpus of long technical manuals. During testing, the LLM frequently produces incomplete answers because relevant information spans multiple retrieved passages, and mid-sentence splits are breaking apart procedural steps. You need to improve retrieval quality before generation without switching the embedding model. What is the most effective change to make?

Accepted Answer

Apply semantic (sentence-aware) chunking with overlap between adjacent chunks during ingestion

Answer

Increase the generation temperature so the model interpolates missing details between passages

Answer

Reduce the number of retrieved documents (top-k) to lower the noise passed to the model

Answer

Switch the index from vector search to keyword-only search to match exact procedural terms