Question

A data engineer is building a Structured Streaming job that reads clickstream events from a cloud storage source and writes a running count of events per product category to a Delta sink. The query uses a groupBy aggregation on the category column. When they attempt to start the stream with outputMode('append'), the job fails to start. Which output mode should they use to correctly write the aggregated results, and why?

Accepted Answer

Use outputMode('complete') because aggregations without a watermark must output the full updated result table on each trigger.

Answer

Use outputMode('append') with a longer trigger interval, since append works for all aggregations once enough time passes.

Answer

Use outputMode('update') because it is the only mode that supports writing to a Delta table sink.

Answer

Keep outputMode('append') but remove the groupBy, since streaming queries cannot perform aggregations.