Question

A data engineer configures a Lakeflow Job with a single task that runs a nightly aggregation. On rare occasions, an upstream data source hangs and the task runs for many hours, blocking the cluster and delaying downstream reports. The engineer wants the job run to automatically stop and be marked as failed if it exceeds 90 minutes, without manually intervening. Which configuration should the engineer apply?

Accepted Answer

Set a timeout (duration threshold) on the job so it is canceled and marked failed after 90 minutes.

Answer

Set the maximum concurrent runs to 1 so the run cannot overlap the next schedule.

Answer

Configure a retry policy with a 90-minute delay between attempts.

Answer

Add a scheduled cron trigger that fires every 90 minutes to interrupt the previous run.