Question

Your team trains a moderately sized image classification model (ResNet-50) on a single Vertex AI custom training job with one NVIDIA A100 GPU. Training is slower than expected, and monitoring shows GPU utilization averaging only 25% with frequent dips to near zero, while CPU utilization is pegged near 100%. Before scaling to more GPUs, what is the most appropriate first action to improve training throughput?

Accepted Answer

Optimize the input pipeline (e.g., use tf.data with parallel map, prefetch, and cached/preprocessed TFRecords) to eliminate CPU-bound data loading bottlenecks

Answer

Immediately switch to a multi-GPU distributed strategy across four A100s to increase overall throughput

Answer

Replace the A100 GPU with a TPU v3 pod slice to accelerate the convolution operations

Answer

Enable mixed-precision training to reduce the GPU memory footprint and speed up matrix operations