Question

Your team runs a daily Cloud Composer (Airflow) DAG that ingests partner sales files, transforms them in Dataproc Serverless, and loads results into BigQuery. Downstream analysts must have refreshed data by 08:00 each morning. Occasionally a partner uploads files late, causing the pipeline to finish after the deadline without anyone noticing until analysts complain. You want Airflow itself to automatically detect when a task takes longer than expected and notify the on-call engineer, without failing the DAG run. What is the most appropriate approach?

Accepted Answer

Define an SLA on the relevant tasks and configure an sla_miss_callback that sends an alert to the on-call channel when the SLA is exceeded.

Answer

Set retries and retry_delay on each task so that late-completing tasks are retried until they finish before 08:00.

Answer

Set an execution_timeout on each task so Airflow marks the task failed and triggers on_failure_callback when it runs long.

Answer

Add a downstream BashOperator that queries the current time and raises an error if it is past 08:00, failing the DAG.