Question

A data analytics team runs interactive Amazon Athena queries against 8 TB of JSON application logs stored in a single Amazon S3 prefix. Analysts complain that queries scanning a few days of data take several minutes and cost far more than expected because each query scans the entire dataset. The team wants to improve query performance and reduce cost with minimal ongoing operational overhead, while continuing to receive newly ingested logs daily. Which combination of changes best achieves this?

Accepted Answer

Convert the logs to Apache Parquet using an AWS Glue ETL job, partition the data by date in S3, and enable Athena partition projection so partitions do not need to be registered in the Glue Data Catalog.

Answer

Increase the Athena workgroup query result reuse window and raise the per-query data scan limit so analysts can scan the full dataset faster.

Answer

Load the JSON logs into an Amazon Redshift cluster and have analysts query Redshift directly instead of Athena.

Answer

Enable S3 Intelligent-Tiering on the log bucket and compress the JSON files with gzip to reduce the amount of data Athena scans.