Question

A data engineer configures a Lakeflow Job with a single task that ingests data from an external API. The API occasionally returns transient HTTP 503 errors that succeed on a subsequent attempt. The engineer wants the task to automatically re-run up to 3 times, waiting 2 minutes between attempts, before the job is marked as failed. Which task configuration should the engineer use?

Accepted Answer

Set a retry policy on the task with maximum retries of 3 and a minimum retry interval of 2 minutes.

Answer

Add three duplicate downstream tasks that each depend on the previous one succeeding.

Answer

Enable the job-level 'Continuous' trigger so the job restarts automatically on failure.

Answer

Configure a 2-minute schedule interval so the job re-runs every 2 minutes.