Question

A media company streams clickstream events into an Amazon MSK cluster. The data engineering team must land these events into an Amazon S3 data lake continuously, partitioned by event date, without writing and maintaining any custom consumer application code. They want a fully managed, scalable connector-based solution that integrates directly with the existing Kafka topics. Which approach best meets these requirements?

Accepted Answer

Deploy an MSK Connect connector using the managed Amazon S3 Sink connector, configuring topic-to-S3 mapping and time-based partitioning

Answer

Write a custom Kafka consumer on an EC2 Auto Scaling group that batches records and uploads them to S3 with date prefixes

Answer

Configure a Lambda function with an MSK event source mapping to read records and put them into S3 objects

Answer

Create a Kinesis Data Firehose delivery stream with the MSK cluster as the source and enable dynamic partitioning to S3