Question

A Generative AI Engineer is deploying two applications to Databricks Model Serving. The first is a lightweight sklearn classifier that returns fraud probabilities, expected to receive fewer than 5 requests per second. The second is a custom fine-tuned 7B-parameter transformer model that must generate text completions at interactive latency. The engineer wants to minimize cost while meeting the performance needs of each workload. Which endpoint configuration approach should the engineer take?

Accepted Answer

Deploy the sklearn classifier on a CPU-based endpoint with a small workload size and scale-to-zero enabled, and deploy the 7B transformer on a GPU-based endpoint sized to meet its latency requirements.

Answer

Deploy both models on the same CPU-based endpoint with a large workload size to share compute and reduce total cost.

Answer

Deploy both models on separate GPU-based endpoints because GPUs always provide lower latency and cost for model serving.

Answer

Deploy the 7B transformer on a CPU-based endpoint with a large workload size, since Model Serving does not support GPU compute for custom models.