Question

Your team has prototyped a gradient-boosted decision tree model (XGBoost) on a small tabular dataset for predicting equipment maintenance needs. You now need to scale training to a 200 GB dataset and later serve batch predictions nightly. A junior engineer suggests provisioning a cluster of GPU-accelerated VMs for both training and inference to maximize speed. What is the most cost-effective and appropriate hardware recommendation for this workload?

Accepted Answer

Use high-memory CPU machines for both distributed training and batch inference, since tree-based models like XGBoost are largely CPU-bound and gain little from GPU acceleration relative to cost.

Answer

Use TPUs for training and GPUs for batch inference, because TPUs offer the best price-performance for all tabular model types.

Answer

Use a large fleet of GPU VMs for training and inference to match the engineer's recommendation, as GPUs accelerate all matrix operations regardless of model architecture.

Answer

Use TPU Pods for training because the 200 GB dataset requires the parallelism only TPUs provide, then serve inference on CPUs.