Question

Your team trains a TensorFlow image classification model on a Vertex AI single-GPU instance. The dataset is 200 GB of TFRecords stored in Cloud Storage and does not fit in memory. During profiling you notice the GPU sits idle between steps, and each epoch reads the data from Cloud Storage anew. You want to maximize GPU utilization and reduce redundant reads without changing hardware. Which tf.data pipeline configuration should you apply?

Accepted Answer

Apply cache() after the map() decode step (caching to local SSD), then shuffle() with a large buffer, batch(), and prefetch(tf.data.AUTOTUNE) with interleave for parallel file reads.

Answer

Call cache() in memory at the very end of the pipeline after batch() so entire batches are held in RAM across epochs.

Answer

Remove prefetch() and increase batch size dramatically so fewer, larger reads occur from Cloud Storage each step.

Answer

Set num_parallel_calls to 1 in map() to preserve deterministic ordering, and rely on the OS page cache to avoid repeated Cloud Storage reads.