Question

A nightly pipeline uses a Copy activity to load a CSV file from a landing folder into a Warehouse table. Occasionally a few rows contain malformed data type values (for example, text in a numeric column), which causes the entire Copy activity to fail and the whole load to abort. Business stakeholders would rather load the valid rows and review the problematic ones later, without failing the run. What should you configure on the Copy activity to meet this requirement with the least effort?

Accepted Answer

Enable fault tolerance to skip incompatible rows and configure a log/redirect location to capture the skipped rows for later review.

Answer

Add a retry policy with a retry count of 3 and a 60-second interval on the Copy activity.

Answer

Replace the Copy activity with a Dataflow Gen2 that removes rows with errors during transformation.

Answer

Wrap the Copy activity in a Try/Catch pattern using an On Failure path that reruns the load from the beginning.