Question

A data engineering team maintains a daily reporting pipeline. A Glue job writes curated Parquet data to Amazon S3 each morning at 06:00 UTC, and a Glue crawler updates the Data Catalog afterward. Business analysts use an Amazon QuickSight dashboard backed by a SPICE dataset built from an Athena query. Analysts report that the dashboard shows yesterday's numbers even though the morning pipeline completed successfully. The team wants the dashboard to reflect the latest data automatically each day with minimal ongoing management. What should the team do?

Accepted Answer

Schedule a SPICE dataset refresh in QuickSight to run after the pipeline completes, ensuring the query re-reads the latest Athena data each morning.

Answer

Switch the QuickSight dataset from SPICE to direct query mode so it always reads the latest S3 data without any refresh configuration.

Answer

Increase the SPICE capacity allocated to the account so that the dataset holds more recent data.

Answer

Re-run the Glue crawler more frequently throughout the day to keep the Data Catalog partitions current.