Question

Your team is scaling a TensorFlow image classification prototype into a production training job on Vertex AI custom training. The training dataset is 4 TB of TFRecord files stored in a Cloud Storage bucket in region us-central1. During initial runs on a single a2-highgpu-4g (4x A100) machine, GPU utilization hovers around 25%, and profiling shows the accelerators frequently idle while waiting for data. The data pipeline already uses tf.data with prefetching. Which change is MOST likely to increase throughput while keeping costs reasonable?

Accepted Answer

Ensure the Cloud Storage bucket, the Vertex AI training job, and any staging resources are all in the same region (us-central1), and use parallel interleaved reads across TFRecord shards with a larger read parallelism setting.

Answer

Switch the training machine to a TPU v3 Pod slice and rewrite the model in JAX to eliminate the input bottleneck.

Answer

Enable mixed-precision training with loss scaling so the A100 GPUs process each batch faster.

Answer

Increase the number of A100 GPUs to 8 by moving to an a2-highgpu-8g machine to double compute capacity.