Question

A media company collects clickstream events from mobile apps into an Amazon Kinesis Data Stream. A data engineer must aggregate events into 5-minute tumbling windows, compute per-user session counts, and write the aggregated results to an S3 data lake in Parquet format. The transformation logic requires stateful windowing that persists across records, and the team wants a managed streaming solution with minimal cluster administration. Which approach best meets these requirements?

Accepted Answer

Use an AWS Glue streaming ETL job that reads from the Kinesis Data Stream, applies a windowed aggregation with Spark Structured Streaming, and writes Parquet output to S3.

Answer

Configure a Kinesis Data Firehose delivery stream with a Lambda transform to buffer and aggregate records before delivering Parquet files to S3.

Answer

Attach a Lambda function directly to the Kinesis Data Stream and accumulate session counts in the function's in-memory state before writing to S3.

Answer

Create an Amazon Athena scheduled query that reads raw JSON delivered by Firehose and produces 5-minute aggregations written back to S3.