Question

A retail company ran a successful gen AI pilot for product description generation using a premium model. Finance approved a budget based on the pilot's monthly costs. Six months after full rollout, actual costs are three times the projected budget, even though usage grew only modestly. The leadership team wants to understand the most likely root cause of the cost overrun so they can plan future initiatives more accurately. What is the MOST likely explanation?

Accepted Answer

The pilot budget only accounted for per-request inference costs and overlooked ongoing costs such as monitoring, human review, data pipeline maintenance, and prompt/output token growth at scale.

Answer

Premium foundation models automatically increase their per-token price once an organization exceeds a fixed number of monthly requests.

Answer

Gen AI models require full retraining every month, and retraining costs were excluded from the pilot budget.

Answer

The company should have used a predictive ML model instead, since generative models cannot be budgeted for at all.