Question

You have a Vertex AI Pipeline built with the Kubeflow Pipelines (KFP v2) SDK. The pipeline trains a model, evaluates it, and conditionally uploads it to the Model Registry only if accuracy exceeds a threshold. Occasionally the training component fails partway through, leaving temporary artifacts in a staging Cloud Storage bucket. The team wants two behaviors: (1) skip the upload step when accuracy is below the threshold, and (2) always run a cleanup step that deletes the staging artifacts, regardless of whether any upstream component succeeds or fails. Which combination of KFP constructs correctly implements both requirements?

Accepted Answer

Wrap the upload logic in a dsl.Condition block keyed on the evaluation output, and register a cleanup component with dsl.ExitHandler at the pipeline level.

Answer

Wrap both the upload and cleanup logic in the same dsl.Condition block, so both run only when accuracy exceeds the threshold.

Answer

Use dsl.ParallelFor to run the upload and cleanup steps concurrently, relying on component caching to skip the upload when accuracy is low.

Answer

Place the cleanup step downstream of the upload step with an explicit .after() dependency, and gate the upload with a try/except in the training component.