Question

A Generative AI Engineer deploys a RAG chatbot on a Model Serving endpoint with inference tables enabled. Two weeks after launch, finance flags that the endpoint's monthly cost has tripled, even though the number of daily requests has stayed roughly flat. The engineer needs to determine the root cause before proposing a fix. Which analysis of the inference table data will most directly identify why costs increased despite stable request volume?

Accepted Answer

Aggregate the total input and output token counts per request over time and compare recent daily averages to the launch-week baseline

Answer

Compute the p95 request latency per day and check whether responses are being served more slowly than at launch

Answer

Measure the embedding drift of user queries against the launch-week baseline distribution using Lakehouse Monitoring

Answer

Count the number of distinct users querying the endpoint and correlate that with the billing period