Question

A financial research firm runs a RAG application on Amazon Bedrock Knowledge Bases backed by an OpenSearch Serverless vector store. Analysts complain that although the correct source documents exist in the knowledge base, the top-retrieved chunks passed to the model are often only tangentially relevant, causing the foundation model to produce incomplete answers. The team already retrieves the top 25 chunks by vector similarity and passes the top 5 to the model. They want to improve the relevance of the 5 chunks the model actually receives without re-ingesting or re-chunking their data. What should they do?

Accepted Answer

Add a reranking model to the retrieve-and-generate flow to reorder the 25 retrieved chunks by relevance and forward the highest-scoring ones to the foundation model.

Answer

Reduce the number of retrieved chunks from 25 to 5 so that only the closest vector matches are considered.

Answer

Switch the OpenSearch index distance metric from cosine similarity to Euclidean distance to sharpen similarity scoring.

Answer

Increase the foundation model's temperature so it can better reason over the loosely relevant chunks.