🔥 3-day streak
AWS Certified Data Engineer - Associate30 / 159
Question 30 of 159

A retail company runs a daily AWS Glue ETL job that reads new CSV files landing in an S3 prefix, converts them to Parquet, and writes to a curated bucket. As data volume grows, the team notices the job reprocesses all files in the source prefix every run, increasing runtime and cost. They want the job to process only files that have not been processed in previous runs, without manually tracking file names or maintaining a separate state store. Which approach best meets this requirement?

Reviewed for accuracy · Report an issueNext question