Question

A Generative AI Engineer logs a custom RAG chain as an MLflow PyFunc model and deploys it to a Model Serving endpoint. During local testing the chain worked perfectly, but the serving endpoint fails to start with an import error for a third-party reranking library that was installed in the development notebook via a cluster library. What is the most likely root cause and the correct fix?

Accepted Answer

The reranking library was available on the cluster but was not captured in the logged model's dependencies; re-log the model with the library included in pip_requirements (or an accurate inferred environment) so the serving container installs it.

Answer

Model Serving endpoints cannot use any third-party libraries; the reranking step must be rewritten using only built-in Databricks functions before deployment.

Answer

The endpoint failed because scale-to-zero was enabled; disabling scale-to-zero will resolve the missing import error.

Answer

The model signature was not defined; adding an input/output signature will cause the serving container to install the missing library automatically.