Question

A Generative AI Engineer is building a RAG application over a knowledge base assembled by scraping the company's public support website. During testing, retrieved chunks frequently contain navigation menus, cookie-consent banners, repeated footer text, and broken HTML tags, which pollute the context passed to the LLM and degrade answer quality. The engineer wants to improve retrieval and generation quality before embeddings are generated. Which action best addresses the root cause?

Accepted Answer

Add a preprocessing step that strips HTML boilerplate (navigation, banners, footers) and extracts clean main-content text before chunking and embedding.

Answer

Increase the chunk size so that the meaningful content dominates each chunk and dilutes the boilerplate noise.

Answer

Switch to a higher-dimensional embedding model that can better distinguish relevant text from noise.

Answer

Add a reranker after retrieval to push the boilerplate-heavy chunks lower in the results.