Question

A retail analytics team streams clickstream events through Amazon Kinesis Data Firehose to an S3 data lake. Business analysts complain that new data takes too long to appear in S3 during low-traffic overnight hours, but during peak daytime traffic the delivery is timely. The team wants near-real-time delivery around the clock without significantly increasing per-object overhead when traffic is heavy. Which configuration change should they make?

Accepted Answer

Lower the Firehose buffer interval (e.g., to 60 seconds) so data is flushed on time even when the buffer size threshold is not reached during low traffic.

Answer

Increase the Firehose buffer size to 128 MB so buffers fill and flush more frequently.

Answer

Replace Firehose with a Kinesis Data Streams consumer running on a fixed EC2 fleet that writes to S3 every few seconds.

Answer

Enable dynamic partitioning on Firehose to speed up delivery during low-traffic periods.