Question

A Generative AI Engineer has built a RAG chain that queries a Databricks Vector Search index and calls a Foundation Model API endpoint. When they log the chain with MLflow using the models-from-code approach and deploy it to a Model Serving endpoint, the endpoint starts but every inference request fails with authentication errors when the chain tries to reach the Vector Search index and the LLM endpoint. What should the engineer do to fix this before re-logging the model?

Accepted Answer

Declare the Vector Search index and the serving endpoint in the 'resources' parameter of mlflow.langchain.log_model so Databricks provisions short-lived credentials for those dependencies at serving time.

Answer

Hardcode a personal access token into the chain's Python source file so the deployed model can authenticate to both dependencies.

Answer

Increase the endpoint's compute size and enable scale-to-zero so the model has enough resources to establish authenticated connections.

Answer

Set the workspace default catalog and schema in the MLflow experiment tags so the serving endpoint inherits the correct permissions.