Question

You maintain a Vertex AI Pipeline built with the Kubeflow Pipelines SDK. It contains several lightweight Python function-based components that each import pandas and scikit-learn. Recently, a pipeline run failed because one component silently pulled a newer scikit-learn version than the one used during development, causing an incompatible model artifact. You want to guarantee that every future run of these components executes with the exact same library versions, while keeping component authoring simple and reproducible. What is the best approach?

Accepted Answer

Build a custom container image with the pinned library versions, push it to Artifact Registry, and reference it as the base_image (and packages_to_install where needed) for each function-based component.

Answer

Add a pip install command with unpinned package names inside each component function so dependencies are freshly resolved at runtime.

Answer

Enable pipeline component caching so the previously successful execution is always reused instead of re-running the component.

Answer

Increase the machine type and memory for the affected component so the newer library version has enough resources to run correctly.