Question

You operate a RAG solution for a legal firm. The knowledge base contains contracts full of specialized legal terminology and clause references. Users report that retrieval frequently returns generic passages instead of the precise clauses they ask about, even after you tuned chunk size and similarity threshold. Analysis shows the general-purpose embedding model places semantically distinct legal terms too close together in vector space. You want the most durable improvement to retrieval accuracy for this domain. What should you do?

Accepted Answer

Fine-tune the embedding model on domain-specific query/passage pairs from the legal corpus so it better separates legal concepts in vector space.

Answer

Increase the number of retrieved chunks (top-k) returned to the LLM so relevant clauses are more likely to be included.

Answer

Lower the similarity threshold further so more candidate passages pass the retrieval filter.

Answer

Switch the generation model to one with a larger context window so more retrieved text can be passed in.