Question

A Generative AI Engineer has deployed a RAG-based customer support assistant with inference tables enabled to capture request and response payloads. Over the past month, users have started asking about a newly launched product line that did not exist when the app was first evaluated. The engineer wants to detect that the distribution of incoming questions has shifted away from the topics covered in the original evaluation set, so they can decide when to refresh the knowledge base and re-run evaluation. Which approach best enables ongoing detection of this shift?

Accepted Answer

Attach a Lakehouse Monitoring inference profile to the inference table, define the original evaluation topic distribution as the baseline, and monitor for drift in the incoming request features.

Answer

Configure Lakehouse Monitoring on the inference table using a snapshot profile that only tracks row counts per day.

Answer

Increase the LLM-as-judge frequency to score every production response for correctness in real time.

Answer

Enable route-optimized serving on the endpoint to reduce latency for the new product questions.