Question

A data science team stores raw event logs as compressed JSON in Amazon S3. They repeatedly run the same complex SQL aggregation in Amazon Athena to build a feature table for model training, and each run scans hundreds of gigabytes, causing high query costs and slow iteration. They want a repeatable, cost-efficient way to persist the engineered feature dataset in an analytics-friendly format for downstream SageMaker training jobs. Which approach best meets these requirements with the least ongoing operational overhead?

Accepted Answer

Use an Athena CREATE TABLE AS SELECT (CTAS) statement to materialize the aggregated features into a new S3 location as partitioned Parquet, then point training jobs at that output.

Answer

Increase the Athena query result cache retention and rerun the same SELECT query before each training job so results are served from cache.

Answer

Export the query results manually from the Athena console to CSV and upload the file to the training bucket before each run.

Answer

Provision an Amazon Redshift cluster, load the raw JSON, and run the aggregation there for every training iteration.