Question

A production Dataflow streaming pipeline reads from Pub/Sub, enriches events, and writes to BigQuery. You discover a bug in the enrichment transform that is silently dropping a subset of valid records. You have fixed the code, but the pipeline is processing millions of events per hour and business requires no gap in processing and no duplicate reprocessing of already-correct output. The pipeline's transform DAG shape is unchanged. What is the best way to deploy the fix while maintaining continuity?

Accepted Answer

Launch the corrected pipeline with the --update flag using the same job name, allowing in-flight state and unacknowledged messages to migrate to the new job.

Answer

Cancel the running job immediately and submit the corrected pipeline as a brand-new job reading from the same subscription.

Answer

Drain the running job, then deploy the corrected pipeline as a new job reading from the same subscription.

Answer

Create a snapshot of the job, delete the pipeline, and restore the snapshot into the corrected pipeline template.