Question

A data engineering team uses AWS Glue DataBrew to interactively clean and transform a customer dataset for a churn-prediction model. After building and validating a sequence of cleaning steps in a DataBrew project, they now need to apply the exact same transformations automatically to new monthly data files that arrive in Amazon S3, without repeating the interactive work each month. What is the MOST appropriate way to operationalize these transformations?

Accepted Answer

Publish the project's recipe and create a DataBrew recipe job that runs against the new S3 dataset, scheduling it to run monthly.

Answer

Export the DataBrew project as a Python script and run it manually in a SageMaker notebook each month.

Answer

Create a DataBrew profile job on the new data each month to reproduce the transformations.

Answer

Recreate the interactive DataBrew project each month and re-enter the transformation steps on the new files.