Question

A data engineering team runs a nightly Glue ETL job that writes curated sales data to Amazon S3, cataloged in the Glue Data Catalog. Business analysts consume this data through an Amazon QuickSight dashboard built on a direct Athena query dataset (not SPICE). After a recent change, analysts report that the dashboard sometimes shows partial data for the current day, appearing before the nightly job completes. The team wants QuickSight to reliably reflect only fully processed data and to refresh automatically once each night's ETL finishes. Which approach best meets this requirement?

Accepted Answer

Add an EventBridge rule triggered on Glue job SUCCEEDED state that invokes a Lambda function to publish a partition-complete marker, and point the QuickSight Athena dataset at a table that only exposes fully committed partitions.

Answer

Switch the QuickSight dataset to SPICE and schedule an hourly SPICE refresh so the dashboard picks up new data throughout the day.

Answer

Increase the Glue job DPU allocation so the ETL completes faster and analysts are less likely to query the table mid-run.

Answer

Enable QuickSight anomaly detection on the dataset so partially loaded days are automatically flagged and hidden from viewers.