Question

A Generative AI Engineer is building an internal prototype chat assistant on Databricks. The traffic is sporadic and unpredictable during the exploration phase, and the team wants to experiment quickly with a state-of-the-art open model (Llama) without committing to reserved capacity or managing infrastructure. They want the lowest operational overhead and to pay only for what they actually use. Which Foundation Model API option should they choose?

Accepted Answer

Use pay-per-token Foundation Model APIs, which expose pre-configured base model endpoints billed per token consumed

Answer

Provision a dedicated provisioned throughput endpoint sized for peak concurrency

Answer

Register the model in Unity Catalog and deploy it to a custom Model Serving endpoint with autoscaling to zero disabled

Answer

Configure an external model endpoint pointing to a self-hosted Llama server outside Databricks