Question

A team builds a RAG application using Amazon Bedrock. They generated and stored document embeddings in a vector database using one embedding model. Later, they switched the query-time embedding model to a different, newer model to improve results, but kept the previously indexed document vectors unchanged. Users now report that retrieved passages are irrelevant to their questions. What is the most likely cause?

Accepted Answer

The query embeddings and stored document embeddings were produced by different models, so they occupy incompatible vector spaces and cannot be compared meaningfully.

Answer

The temperature parameter on the generation model was set too high, causing random retrieval.

Answer

The context window of the foundation model is too small to hold the retrieved passages.

Answer

The vector database does not support enough dimensions for the newer model's output.