Question

A retail company streams IoT sensor readings from thousands of devices through Pub/Sub into a Dataflow streaming pipeline. Each message contains a device-generated event timestamp. Network conditions cause some devices to deliver messages several minutes late and out of order. The team wants aggregations grouped by the actual time the event occurred at the device, not the time Dataflow received the message. Which approach correctly configures the pipeline to reason about event time?

Accepted Answer

Configure the Pub/Sub read to extract the device event timestamp as the record timestamp attribute so Dataflow advances the watermark against event time.

Answer

Assign the Pub/Sub message publish time as the record timestamp and use processing-time triggers to emit results immediately.

Answer

Disable watermarks entirely and buffer all messages in a global window until the pipeline is drained.

Answer

Use Bigtable as an intermediate sink to reorder messages by timestamp before feeding them back into Dataflow.