Question

A Generative AI Engineer is building a RAG application on Databricks. They need to generate embeddings for both the initial batch indexing of 500,000 documents and for real-time query embeddings at inference time. They want to minimize operational overhead by using a Databricks-hosted embedding model rather than deploying a custom model. Which approach correctly uses the Databricks Foundation Model APIs for this task?

Accepted Answer

Call the pay-per-token Foundation Model API embedding endpoint (e.g., databricks-bge-large-en) for both batch indexing and query-time embedding, ensuring the same model is used in both paths.

Answer

Use the pay-per-token embedding endpoint for real-time queries, but use a fine-tuned chat completion endpoint to generate embeddings during batch indexing.

Answer

Register an external model endpoint pointing to a third-party chat completion API and use it to produce embeddings for both indexing and queries.

Answer

Use the Foundation Model API text generation endpoint (e.g., DBRX Instruct) to embed documents, since instruction-tuned models produce higher-quality vectors.