Question

A GenAI engineer has three separate registered models in Unity Catalog: a small classifier that routes queries, a RAG chain for document questions, and a summarization model. Leadership wants to minimize the number of serving endpoints to reduce operational overhead, but each model must be independently scalable and callable. Which approach best satisfies these requirements on Databricks Model Serving?

Accepted Answer

Serve all three models by adding them as separate served entities on a single Model Serving endpoint, each with its own scaling configuration and traffic percentage.

Answer

Create one endpoint per model since a Databricks Model Serving endpoint can only host a single served entity at a time.

Answer

Log all three models into a single MLflow pyfunc wrapper and deploy one served entity, routing internally with conditional logic.

Answer

Deploy the three models to a single endpoint using A/B traffic splitting so requests are randomly distributed across the models.