Question

A Generative AI Engineer must embed 4 million product-description records stored in a Delta table before creating a Databricks Vector Search index. They plan to call a Databricks Model Serving endpoint hosting an embedding model. Initial testing shows the job is extremely slow because each row is sent to the endpoint one at a time in a Python loop. Which change will most improve embedding throughput while remaining a supported Databricks pattern?

Accepted Answer

Use ai_query or a pandas_udf to send batched requests of many records per call, leveraging Spark parallelism across partitions.

Answer

Use a Spark UDF that sends one record per invocation, relying on cluster autoscaling to add more single-request executors.

Answer

Switch to a Delta Sync Vector Search index so the embeddings are computed sequentially by the sync process without any batching.

Answer

Reduce the dataset to 100,000 rows so the single-row loop completes in an acceptable amount of time.