Question

A data engineering team runs a nightly Glue ETL pipeline orchestrated by AWS Step Functions. Business rules require that after the transformation step completes and Glue Data Quality checks pass, a data steward must manually review a summary report before the curated data is published to the production S3 prefix that feeds Athena and QuickSight. The team wants the workflow to pause and wait for the steward's explicit approval, then continue automatically, without polling or wasting compute. What is the MOST appropriate way to implement this pause-and-approve behavior?

Accepted Answer

Use a Step Functions callback pattern with the .waitForTaskToken integration, sending an SNS/email notification containing the task token, and have the steward's approval action invoke SendTaskSuccess to resume the workflow

Answer

Add a Wait state configured for 12 hours in the state machine to give the steward time to review, then automatically proceed to the publish step

Answer

Split the pipeline into two separate Step Functions state machines and have the steward manually start the second (publish) machine after review

Answer

Configure a Lambda function that polls a DynamoDB approval table every minute inside a loop until an approval flag is set, then continue the workflow