Question

A media company already streams clickstream events into an Amazon MSK (provisioned) cluster from its application servers. The data engineering team must land these events in Amazon S3 as newline-delimited JSON for downstream Athena queries, with the least amount of custom code and no self-managed consumer infrastructure to operate. Which approach best meets these requirements?

Accepted Answer

Configure an Amazon Data Firehose delivery stream that uses the MSK cluster as its source and Amazon S3 as its destination.

Answer

Deploy a fleet of EC2 instances running a Kafka consumer group that reads from the MSK topic and writes objects to S3.

Answer

Create an AWS Lambda function with an MSK event source mapping that batches records and uses the S3 PutObject API for each invocation.

Answer

Use AWS DMS with the MSK cluster as the source endpoint and an S3 target endpoint to replicate the streaming events.