Question

A financial analytics company runs steady daily inference workloads on an on-premises cluster of 8 GPUs. Roughly once per quarter, the data science team needs to train several large models simultaneously for a two-week period, requiring far more GPU capacity than the on-premises cluster can provide. Management wants to avoid purchasing hardware that would sit idle most of the year. Which infrastructure strategy best fits these requirements?

Accepted Answer

Keep the steady inference workloads on-premises and use a hybrid model that bursts the periodic training jobs to GPU instances in the cloud

Answer

Migrate all workloads, including daily inference, entirely to the cloud to simplify management

Answer

Purchase enough additional on-premises GPUs to cover the peak quarterly training demand

Answer

Reduce the size of the training models so they fit within the existing 8-GPU on-premises cluster