Question

A data engineering team runs a nightly Glue workflow that ingests transactional files, transforms them, and updates the Data Catalog. The workflow must start automatically at 02:00 UTC every day. During a recent incident, an upstream system delivered files late, and the team wants a way to also trigger the same workflow manually when they know files have arrived early, without duplicating the workflow definition. Which approach meets both requirements with the least operational overhead?

Accepted Answer

Configure a scheduled trigger on the workflow for 02:00 UTC and start the same workflow on demand via the StartWorkflowRun API or console when needed.

Answer

Create two separate Glue workflows with identical job definitions, one with a scheduled trigger and one with an on-demand trigger.

Answer

Use an EventBridge rule with a cron expression as the only trigger, and disable Glue triggers entirely so the workflow can only run on a schedule.

Answer

Convert the workflow jobs into a single Glue job with a cron schedule, and re-run the job manually by editing its schedule to the current time each morning.