Question

You are deploying a RAG chatbot to a Databricks Model Serving endpoint. The application is used sporadically during business hours, with long idle periods overnight and on weekends. Finance wants to minimize compute cost, but the product team insists that occasional first-request latency spikes after idle periods are acceptable for this internal tool. Which endpoint configuration best satisfies both requirements?

Accepted Answer

Enable scale-to-zero so the endpoint releases compute during idle periods and provisions capacity on the next request.

Answer

Set a fixed minimum of 4 provisioned concurrency units to guarantee no latency spikes at any time.

Answer

Disable autoscaling entirely and use a single always-on small instance sized for peak traffic.

Answer

Deploy the model as a batch inference job scheduled to run every hour instead of a real-time endpoint.