Question

Your team runs a GKE Standard cluster for a web application. You need to add a machine-learning inference service that requires NVIDIA GPUs, but you do not want the existing CPU-only web pods to consume the more expensive GPU nodes. What should you do?

Accepted Answer

Create a new node pool with GPU-attached machine types; Kubernetes automatically applies a taint to the GPU nodes so only pods that tolerate it and request GPUs are scheduled there.

Answer

Resize the existing default node pool to use GPU machine types and let the scheduler distribute all pods across the nodes.

Answer

Deploy the inference service to Cloud Run instead, since GKE does not support GPUs.

Answer

Create a second cluster with GPU nodes and manually move the web application pods there when GPUs are idle.