Question

A media company streams clickstream events into Amazon Kinesis Data Firehose, which delivers them to Amazon S3. Before delivery, each JSON record must be cleansed: malformed records dropped, a timestamp field normalized to ISO 8601, and a derived 'session_bucket' field added. The volume is moderate (a few thousand records/second), and the team wants a fully managed, low-operational solution that transforms records in-flight without standing up clusters. Which approach best meets these requirements?

Accepted Answer

Attach an AWS Lambda function to the Firehose delivery stream as a data transformation, returning each record with a processing result status of Ok, Dropped, or ProcessingFailed.

Answer

Deliver raw records to S3 first, then run a scheduled AWS Glue ETL job to cleanse, normalize, and enrich the data into a curated bucket.

Answer

Launch a transient Amazon EMR cluster with Spark Streaming that reads directly from Firehose to transform records before writing to S3.

Answer

Configure a Kinesis Data Analytics (Managed Service for Apache Flink) application to consume from Firehose and apply the transformations.