Question

Your data engineering team runs ad hoc Spark analytics on several long-lived Dataproc clusters. During a cost review, you discover that analysts frequently leave clusters running overnight and on weekends after their jobs complete, generating significant idle compute charges. You need a repeatable, low-maintenance way to eliminate these idle costs without requiring analysts to remember to shut clusters down or changing how they submit jobs. What should you do?

Accepted Answer

Configure a Dataproc cluster scheduled deletion policy that automatically deletes each cluster after a specified idle timeout period.

Answer

Add a Cloud Monitoring alert that emails the team whenever cluster CPU utilization drops below 5% for more than an hour.

Answer

Switch all clusters to use only preemptible secondary workers so idle time is billed at the discounted spot rate.

Answer

Enable Dataproc autoscaling with a scale-down policy so idle clusters gradually reduce their worker count to the minimum.