🔥 3-day streak
Professional Machine Learning Engineer145 / 166
Question 145 of 166

Your team runs a Vertex AI Pipeline daily. The pipeline has a data-validation step, a heavy preprocessing step (transforming a static reference dataset that rarely changes), and a training step whose hyperparameters you tune frequently. During tuning iterations, the input data and preprocessing code are unchanged, but each full pipeline run re-executes the expensive preprocessing, wasting time and cost. You want to minimize redundant computation across runs while ensuring the step re-runs automatically if its inputs or code change. What is the most appropriate approach?

Reviewed for accuracy · Report an issueNext question