Question

A Generative AI Engineer builds a RAG chain locally using LangChain 0.2.x and a specific version of the databricks-vectorsearch SDK. After logging the chain with MLflow and deploying it to a Model Serving endpoint, the endpoint fails to start with import errors that never appeared during local development. The engineer confirms the code is identical. What is the most likely cause and correct fix?

Accepted Answer

The serving environment rebuilt with different library versions than local; explicitly capture and pin dependencies (e.g., via the model's inferred pip_requirements or an extra_pip_requirements list) when logging the model so the serving container matches the development environment.

Answer

Model Serving does not support LangChain chains; the engineer must rewrite the chain as a pyfunc model with no external dependencies before deploying.

Answer

The endpoint needs scale-to-zero disabled so the container stays warm and can resolve imports; enable provisioned concurrency to fix the import errors.

Answer

The Unity Catalog model registration stripped the dependency metadata; re-register the model using the Workspace Model Registry instead of Unity Catalog to preserve libraries.