NVIDIA-Certified Associate: AI Infrastructure and Operations

Question 1

A retail analytics team wants to build a model that predicts whether a customer will churn based on historical account data. They have three years of records where each customer is already tagged as 'churned' or 'retained'. A data scientist argues this is a classic supervised learning problem. Which characteristic of the team's dataset most directly justifies classifying this as supervised learning?

Accepted Answer

The dataset contains labeled examples with known churn outcomes used to train the model to map inputs to correct outputs

Answer

The dataset is large enough to require GPU acceleration for training

Answer

The model must group customers into clusters without any predefined categories

Answer

The model learns by receiving rewards and penalties as it interacts with a live environment

Question 2

A cloud provider is building a multi-tenant AI cluster and wants to offload networking, storage virtualization, and security (encryption, isolation) tasks away from the host CPUs so that more CPU cycles remain available for feeding data to the GPUs. Which NVIDIA component is specifically designed to handle this infrastructure processing offload?

Accepted Answer

NVIDIA BlueField DPU (Data Processing Unit)

Answer

NVIDIA NVSwitch

Answer

NVIDIA Tensor Cores

Answer

NVIDIA Grace CPU Superchip

Question 3

A cloud provider is building a multi-tenant GPU cluster where each node must isolate tenant network traffic, run software-defined networking, and offload storage virtualization from the host CPUs without consuming GPU-adjacent compute resources. The architecture team is choosing between two network adapter options for the compute nodes. Which adapter best meets these requirements?

Accepted Answer

A BlueField DPU, which combines a high-speed NIC with programmable Arm cores to offload networking, storage, and security services from the host

Answer

A standard ConnectX SmartNIC, which provides high-throughput connectivity but relies on the host CPU for SDN and storage virtualization

Answer

A PCIe retimer card, which extends signal integrity across long PCIe traces for additional GPUs

Answer

An NVSwitch module, which provides all-to-all GPU bandwidth within the node

Question 4

A European healthcare company wants to fine-tune a diagnostic model using patient data that, by regulation, must never leave the EU. The data science team prefers a fully managed cloud GPU service to avoid operating their own data center, but the compliance officer flags concerns about where the data is processed and stored. Which factor should most influence the choice of cloud GPU deployment?

Accepted Answer

Selecting a provider and region where compute and storage are guaranteed to reside within EU jurisdiction to meet data residency requirements

Answer

Choosing the region with the lowest per-GPU-hour pricing regardless of physical location

Answer

Using the region that offers the newest GPU architecture even if it is located outside the EU

Answer

Distributing the workload across all available global regions to maximize GPU availability

Question 5

A startup wants to begin training a mid-sized language model within two weeks but has no data center space, limited capital, and uncertain long-term GPU demand that may spike or drop month to month. The team needs multi-GPU nodes with high-speed interconnect but cannot commit to a multi-year hardware purchase. Which infrastructure approach best fits these constraints?

Accepted Answer

Rent GPU instances from a cloud provider using on-demand or short-term reserved capacity

Answer

Purchase an on-premises DGX H100 system and colocate it in a rented data center rack

Answer

Build a custom multi-node cluster from individually sourced PCIe GPUs and commodity servers

Answer

Deploy a DGX SuperPOD scalable unit under a three-year capital lease

Question 6

A data center team is specifying a new GPU server for training computer-vision models. The GPUs each have 80 GB of HBM, and the workload streams large batches of augmented image data. During a pilot, engineers notice the data-loading and augmentation stage frequently stalls because the OS cannot cache enough of the working dataset and swapping occurs. The GPUs remain underutilized. Which infrastructure adjustment most directly addresses this specific bottleneck?

Accepted Answer

Increase the amount of system (host) RAM so the data pipeline and OS page cache can hold more of the working dataset

Answer

Add more GPUs to the server to raise aggregate HBM capacity

Answer

Replace the InfiniBand adapters with higher-bandwidth NVLink bridges

Answer

Switch the training precision from FP32 to FP16 to reduce GPU memory pressure

Question 7

A data science team runs a computer-vision training pipeline on a multi-GPU server. During training, GPU utilization repeatedly dips to 40-50%, while the host CPUs sit pinned near 100% performing JPEG decode and augmentation for the input pipeline. Storage bandwidth and network are well within limits. What is the most appropriate infrastructure adjustment to improve GPU utilization?

Accepted Answer

Increase the CPU-to-GPU balance by provisioning more CPU cores (or offloading decode/augmentation to the GPU) so the data pipeline can keep the GPUs fed

Answer

Add faster NVMe storage to the server to raise read throughput

Answer

Replace the GPUs with higher-memory GPU models to fit larger batches

Answer

Add an InfiniBand adapter to increase inter-node bandwidth

Question 8

A data science team is evaluating why their newly acquired NVIDIA GPU dramatically outperforms their old CPU cluster on a deep learning training job. During a design review, a colleague asks specifically what architectural characteristic of the GPU is most responsible for accelerating the dense matrix multiplications at the core of neural network training. Which explanation is most accurate?

Accepted Answer

The GPU contains thousands of relatively simple cores that execute many arithmetic operations concurrently, exploiting the high degree of data parallelism in matrix math.

Answer

The GPU runs at a much higher clock frequency than any CPU, so each individual instruction completes far faster.

Answer

The GPU offloads work to the operating system scheduler, which spreads threads more efficiently than a CPU can.

Answer

The GPU uses a larger last-level cache than the CPU, eliminating the need to access memory during training.

Question 9

A data science team hands off a trained deep learning model to operations for deployment. The operations engineer provisions a standard server that has powerful multi-core CPUs but no GPU, and installs the CUDA Toolkit expecting to accelerate inference. The application refuses to run with a CUDA initialization error. What is the fundamental reason for this failure?

Accepted Answer

CUDA is a parallel computing platform that requires an NVIDIA GPU present in the system; without compatible GPU hardware there is nothing for CUDA to execute on.

Answer

The CUDA Toolkit version was too new and must be downgraded to match the CPU microarchitecture.

Answer

CUDA runs fine on CPUs, but the cuDNN library was missing, which is what actually enables CPU inference.

Answer

The model needed to be re-exported to ONNX before CUDA could run it on the CPU cores.

Question 10

A data science team is scaling a large deep learning training job across 8 GPUs inside a single server. They observe that as they add more GPUs, training throughput does not increase proportionally, and profiling shows significant time spent transferring gradient data between GPUs over the standard PCIe bus. Which NVIDIA technology is specifically designed to address this GPU-to-GPU communication bottleneck?

Accepted Answer

NVLink, a high-bandwidth interconnect for direct GPU-to-GPU data transfer

Answer

cuDNN, a library of optimized deep learning primitives

Answer

Triton Inference Server, for serving trained models at scale

Answer

MIG, which partitions a single GPU into isolated instances

Question 11

A development team is porting a compute-intensive physics simulation to run on NVIDIA GPUs. The engineers want direct, low-level control over how their custom kernels execute across thousands of GPU threads, and they need to write code that explicitly manages parallel execution and GPU memory. Which component of the NVIDIA software stack provides this foundational programming model?

Accepted Answer

CUDA

Answer

Triton Inference Server

Answer

NGC Catalog

Answer

NVIDIA AI Enterprise

Question 12

A data science team wants to ship a deep learning inference application to run on several different customer clusters. Each cluster has different host OS versions and locally installed frameworks, and the team is spending significant effort resolving mismatched library dependencies during every deployment. They want a repeatable way to package the application with all its CUDA, cuDNN, and framework dependencies so it runs consistently across environments. Which NVIDIA software approach best addresses this need?

Accepted Answer

Distribute a GPU-optimized container image from the NGC catalog that bundles the framework, CUDA, and cuDNN libraries

Answer

Ask each customer to manually install the matching CUDA Toolkit and cuDNN versions on their host OS

Answer

Recompile the application from source on every target cluster to match its local libraries

Answer

Upgrade the GPU driver on every cluster to the newest release and rely on that to standardize dependencies

Question 13

A new data scientist on your team is confused about the NVIDIA software stack. They ask which component provides the core programming interface and libraries that allow AI frameworks like PyTorch to execute general-purpose parallel computations directly on the GPU. Which component should you point them to?

Accepted Answer

CUDA — the parallel computing platform and programming model for GPUs

Answer

Triton Inference Server — for deploying trained models in production

Answer

NGC — a catalog of containers and pretrained models

Answer

NVIDIA AI Enterprise — a licensing and support subscription tier

Question 14

A data science team is setting up a new deep learning training environment on NVIDIA GPUs. They already have the CUDA Toolkit installed, but their PyTorch models are running slower than expected during convolution-heavy operations. A colleague suggests ensuring a specific NVIDIA library is properly installed and used by the framework. Which library is specifically designed to provide highly optimized, GPU-accelerated primitives for deep neural network operations such as convolutions, pooling, and activation functions?

Accepted Answer

cuDNN (CUDA Deep Neural Network library)

Answer

NGC (NVIDIA GPU Cloud) container registry

Answer

Triton Inference Server

Answer

NVLink interconnect driver

Question 15

Before dispatching a large multi-node training job on a newly repaired GPU node, an operator wants to run a comprehensive validation that stresses memory, executes compute tests, and checks integration health beyond a quick sanity check. Using the DCGM diagnostic tool (dcgmi diag), which run level should the operator select to perform this deeper, longer-running validation?

Accepted Answer

Run level 3 (-r 3), which includes extended memory and hardware stress tests taking several minutes

Answer

Run level 1 (-r 1), a quick sub-second sanity check of deployment and software configuration

Answer

Run level 4 (-r 4), which disables ECC to speed up throughput benchmarking

Answer

Run level 0 (-r 0), which only reports the current driver and firmware versions

Question 16

During routine fleet monitoring, DCGM reports a steadily increasing count of correctable (single-bit) ECC memory errors on one GPU in a training node. The GPU continues to complete jobs successfully, but the error count grows week over week. What is the most appropriate operations response?

Accepted Answer

Track the trend and allow row remapping/page retirement to handle affected memory locations, drain and investigate only if uncorrectable errors appear or retired pages exceed vendor thresholds

Answer

Immediately RMA the GPU because any correctable ECC error indicates imminent hardware failure and data corruption

Answer

Disable ECC on the GPU to stop the error counter from incrementing and improve available memory bandwidth

Answer

Ignore the metric entirely since correctable errors are automatically fixed and never affect reliability

Question 17

You operate a 40-node GPU cluster and need continuous, fleet-wide visibility into per-GPU utilization, memory usage, temperature, and ECC error counts. Your platform team already runs Prometheus for metrics collection and Grafana for dashboards. Which approach best integrates NVIDIA GPU telemetry into this existing observability stack?

Accepted Answer

Deploy DCGM-Exporter on each node to expose GPU metrics in a Prometheus-scrapable format, then visualize them in Grafana

Answer

Run nvidia-smi in a cron job on each node and parse the text output into log files for manual review

Answer

Enable the GPU driver's built-in SNMP agent and point Prometheus at each node's SNMP endpoint

Answer

Rely on Kubernetes node-exporter, which automatically reports GPU utilization and ECC errors

Question 18

After a routine driver upgrade on a DGX system with NVSwitch, several multi-GPU training jobs fail immediately at startup with NCCL errors, and nvidia-smi shows all GPUs present and healthy. Single-GPU jobs run fine. Checking the host, you find the fabric manager service is not running. What is the most likely cause and correct remediation?

Accepted Answer

The NVIDIA Fabric Manager service failed to start after the upgrade; restart/enable it so NVSwitch is initialized and NVLink peer-to-peer paths are established

Answer

The GPUs have overheated; drain the node and wait for thermal recovery before rescheduling jobs

Answer

MIG mode was enabled during the upgrade; disable MIG on all GPUs to restore multi-GPU communication

Answer

The InfiniBand adapters lost their subnet manager assignment; restart the subnet manager on the fabric

Question 19

A cluster operator notices that several Slurm training jobs intermittently fail minutes after starting, always landing on the same few GPU nodes. The failures do not appear during normal idle periods. The operator wants to automatically prevent jobs from being scheduled onto GPUs that have degraded or failing hardware before the job begins. Which approach best addresses this?

Accepted Answer

Integrate DCGM diagnostics (dcgm health checks) as a prolog step so nodes with unhealthy GPUs are drained or excluded before jobs run

Answer

Increase the Slurm job time limit so failing jobs have more time to complete before timing out

Answer

Reduce the per-job GPU memory request so workloads fit into a smaller partition

Answer

Enable MIG on all affected nodes to isolate tenants from one another

Question 20

A multi-GPU training job on a DGX node suddenly shows much lower throughput than a previous identical run on the same node. GPU utilization (SM activity) and power draw both read near 100%, temperatures are normal, and no XID errors appear in the logs. An administrator wants to confirm whether inter-GPU communication has degraded before escalating a hardware ticket. Which DCGM metric should the administrator examine first?

Accepted Answer

NVLink bandwidth / NVLink flit and CRC error counters

Answer

GPU framebuffer memory used (FB used)

Answer

PCIe replay counter for the host bridge only

Answer

GPU power usage (power draw in watts)

Question 21

You administer a multi-node training cluster. Users report that a specific 8-GPU node occasionally produces corrupted gradient reductions during NCCL all-reduce operations, but only under heavy load. GPU utilization, power draw, and temperature all appear normal in your dashboards. You want to use DCGM to pinpoint whether an intermittent interconnect fault is the root cause. Which DCGM field is most appropriate to investigate first?

Accepted Answer

NVLink CRC (flit/data) error counters per link

Answer

SM clock frequency field to detect throttling

Answer

GPU utilization percentage over the training window

Answer

Framebuffer (memory) used versus total capacity

Question 22

A cluster administrator receives complaints that GPU nodes appear busy but training throughput is low. They want to attribute measured GPU SM activity, memory usage, and energy to individual Slurm jobs so they can identify which jobs underutilize their allocated GPUs. Which approach provides per-job GPU accounting metrics with the least custom development?

Accepted Answer

Enable DCGM job statistics (dcgmi stats / job-level accounting) integrated with the Slurm prolog/epilog to record per-job GPU utilization, memory, and power

Answer

Parse the output of nvidia-smi at 1-second intervals via a cron job and manually correlate timestamps to Slurm job IDs afterward

Answer

Read the GPU fan speed and clock registers with nvidia-settings for each node and estimate utilization from thermal behavior

Answer

Rely on the Slurm sacct CPU efficiency report (seff) to infer GPU utilization from CPU time consumed by each job

Question 23

An operations team wants to track energy efficiency across a large training cluster. They need a per-GPU telemetry field that reports real-time board power draw in watts so they can correlate power consumption with job phases and identify GPUs that are drawing significantly more power than peers under similar load. Which DCGM field group best serves this requirement?

Accepted Answer

Power utilization metrics (e.g., power usage in watts and total energy consumption)

Answer

ECC memory error counters (volatile and aggregate)

Answer

PCIe replay and NVLink CRC error counters

Answer

XID error event fields reported through the device event log

Question 24

You operate a 32-node GPU training cluster. Users report that jobs landing on one specific node run roughly 20% slower than identical jobs on other nodes, but no jobs are failing and no XID errors appear in the logs. You want to confirm whether the GPUs on that node are silently reducing performance before you take any disruptive action. Which DCGM field group should you examine first?

Accepted Answer

Clock throttling reason fields (e.g., thermal and power violation counters) exposed by DCGM

Answer

The ECC aggregate uncorrectable error counters reported by DCGM

Answer

The PCIe replay counter fields reported by DCGM

Answer

The NVLink CRC error count fields reported by DCGM

Question 25

During a large multi-node training run, one node repeatedly crashes its training process. The cluster operator wants to determine whether a specific GPU is experiencing hardware faults such as ECC errors or falling off the bus. Which NVIDIA tool and data source should the operator use to programmatically collect GPU health events, XID error codes, and run active diagnostics before deciding whether to drain the node?

Accepted Answer

NVIDIA DCGM, which exposes health checks, XID error reporting, and diagnostic levels (dcgmi diag) for GPU fault detection

Answer

Kubernetes kubectl describe node, which reports GPU hardware fault codes and ECC error counts directly

Answer

The CUDA Toolkit's nvcc compiler diagnostics, which flag GPU hardware errors at build time

Answer

Slurm's sacct accounting database, which records per-GPU XID codes and thermal fault events

Question 26

A data science team is building an image classifier to detect defective parts from thousands of high-resolution factory photos. A traditional machine learning approach would require engineers to manually design features (edges, textures, shapes) before training a classifier. The team instead chooses a deep learning convolutional neural network. What is the primary advantage that motivates this choice?

Accepted Answer

Deep neural networks automatically learn hierarchical feature representations directly from raw pixel data, removing the need for manual feature engineering.

Answer

Deep learning models always require far less training data than classical machine learning methods.

Answer

Convolutional neural networks eliminate the need for any labeled training examples.

Answer

Deep learning models can be trained effectively on CPUs alone without needing GPU acceleration.

Question 27

A mid-sized enterprise wants to deploy its first on-premises AI cluster of 8 DGX systems for a team of data scientists. Leadership requires a solution that ships as a pre-validated, integrated design covering compute, networking, and storage from certified partners, minimizing the risk and time of designing the cluster from scratch. Which NVIDIA offering best matches this requirement?

Accepted Answer

NVIDIA DGX BasePOD reference architecture

Answer

A collection of individually purchased NVIDIA-Certified Systems assembled in-house

Answer

NVIDIA DGX Cloud subscription

Answer

A single NVIDIA DGX Station workstation

Question 28

A mid-sized enterprise wants to deploy a GPU server for AI training in their own data center. Their IT team lacks deep expertise in validating GPU compute, networking, and thermal designs, and they want assurance that the hardware has been tested to run NVIDIA AI software reliably. They also want flexibility to purchase from their existing OEM hardware vendor rather than buying a full turnkey appliance from NVIDIA. Which option best meets these needs?

Accepted Answer

Purchase an NVIDIA-Certified System from a partner OEM that has passed NVIDIA's validation for performance, manageability, and security

Answer

Purchase an NVIDIA DGX system directly, since it is the only supported way to run NVIDIA AI software

Answer

Build a custom server with consumer GeForce GPUs to minimize cost and rely on community drivers

Answer

Deploy an HGX baseboard as a standalone product and integrate it themselves without vendor validation

Question 29

A research startup needs to begin training large models within two weeks but has no data center, no facilities staff, and no capital budget for hardware. They want a fully managed, NVIDIA-optimized multi-node GPU environment where the software stack and cluster infrastructure are handled for them, so their small team can focus purely on model development. Which consumption model best fits these constraints?

Accepted Answer

Subscribe to NVIDIA DGX Cloud, accessed through a cloud service provider

Answer

Purchase and rack a DGX BasePOD on-premises to guarantee dedicated performance

Answer

Build a custom multi-GPU server from PCIe cards to minimize upfront spend

Answer

Deploy a DGX Station under a desk as a shared workgroup resource

Question 30

An enterprise has deployed a multi-node DGX cluster for a data science team. The infrastructure operations lead needs a single platform to schedule and monitor multi-node training jobs, manage user access to shared GPU resources, and track cluster utilization across the fleet. Which NVIDIA software is purpose-built to provide this job scheduling, resource management, and monitoring for on-premises DGX clusters?

Accepted Answer

NVIDIA Base Command Manager

Answer

NVIDIA Triton Inference Server

Answer

NVIDIA TensorRT

Answer

NVIDIA cuDNN

Question 31

An infrastructure architect is documenting the internal specifications of a new DGX H100 system for the operations team. A colleague asks what role the two host CPUs play in the system, given that the eight H100 GPUs perform the actual deep learning computation. Which statement BEST describes the primary function of the host CPUs in this configuration?

Accepted Answer

They orchestrate the system, run the OS, manage data loading and preprocessing, and feed batches to the GPUs, rather than performing the core tensor math.

Answer

They execute the matrix multiplications for training in parallel with the GPUs to increase overall FLOPS.

Answer

They provide the high-bandwidth memory (HBM) that the GPUs use to store model weights during training.

Answer

They handle the all-to-all GPU collective communication that would otherwise require NVSwitch.

Question 32

A data center operations team is preparing to install a single NVIDIA DGX H100 system (8x H100 SXM GPUs) into an existing rack. The facility's power and cooling team asks the AI infrastructure engineer what the most important electrical planning consideration is for this node. Which statement best reflects the correct planning approach for a DGX H100?

Accepted Answer

A DGX H100 draws roughly 10 kW at peak, so the rack must be provisioned with sufficient high-density power distribution and cooling to sustain it, not just standard office-server circuits.

Answer

A DGX H100 runs entirely on a single standard 120V/15A wall outlet, so no special power distribution planning is needed.

Answer

Power planning is unnecessary because the H100 GPUs automatically underclock to stay within a 1.5 kW node budget regardless of workload.

Answer

The dominant electrical concern is the InfiniBand switch power, since the DGX node itself consumes under 2 kW during training.

Question 33

An AI infrastructure team is configuring a new DGX H100 system. They notice it ships with two types of internal NVMe storage: a smaller mirrored pair intended for the operating system and a larger set of drives configured as a RAID 0 volume. A data scientist asks which internal storage the RAID 0 volume is intended for. What is the correct guidance?

Accepted Answer

The RAID 0 volume is high-speed local scratch space for staging active datasets and cache, prioritizing throughput over redundancy

Answer

The RAID 0 volume is the primary boot and OS drive because mirroring the OS would waste capacity

Answer

The RAID 0 volume is meant for long-term archival of completed training datasets and should be backed up daily

Answer

The RAID 0 volume should host the only copy of critical model checkpoints since RAID protects against drive failure

Question 34

A data center architect is comparing the intra-node GPU interconnect of an NVIDIA DGX H100 system against an older DGX A100 system to justify an upgrade for large-scale model training. She needs to explain to management the primary technical advantage of the newer system's scale-up fabric between the eight GPUs inside a single node. Which statement best describes the key improvement?

Accepted Answer

The DGX H100 uses fourth-generation NVLink and third-generation NVSwitch, providing significantly higher per-GPU NVLink bandwidth for all-to-all GPU communication than the DGX A100

Answer

The DGX H100 replaces NVLink entirely with PCIe Gen5, which delivers higher aggregate bandwidth than any NVLink generation

Answer

The DGX H100 uses InfiniBand NDR between the eight internal GPUs instead of NVLink to reduce switch complexity

Answer

The DGX H100 improves interconnect speed only by increasing system DRAM capacity, leaving the GPU-to-GPU fabric unchanged from the A100

Question 35

An AI infrastructure team is designing a cluster of eight DGX H100 systems for large-model training. Within each DGX node, all eight GPUs must exchange gradients at maximum bandwidth during every training step, and the eight nodes must also communicate for multi-node all-reduce operations. Which interconnect strategy correctly matches each communication requirement?

Accepted Answer

Use NVLink/NVSwitch for GPU-to-GPU communication inside each node, and InfiniBand for communication between the eight nodes.

Answer

Use InfiniBand for GPU-to-GPU communication inside each node, and NVLink for communication between the eight nodes.

Answer

Use standard PCIe for intra-node GPU communication and 10GbE Ethernet for inter-node communication to reduce cost.

Answer

Use NVLink for both intra-node and inter-node communication by cabling NVLink directly between separate DGX chassis.

Question 36

A data center team is deploying a cluster of DGX H100 systems for large-scale distributed training. During network fabric planning, they want to ensure that high-volume dataset reads from the shared parallel file system do not compete with the GPU-to-GPU gradient exchange traffic during collective operations. What is the recommended approach on the DGX H100 platform to achieve this separation?

Accepted Answer

Use dedicated InfiniBand adapters for the compute (east-west) fabric and separate network adapters for the storage fabric

Answer

Route both storage and compute traffic over the same set of ConnectX-7 adapters and rely on QoS priority queues to arbitrate

Answer

Send all gradient exchange traffic over NVLink and all storage traffic over a single 10GbE management port

Answer

Disable GPUDirect RDMA so storage and compute traffic are serialized through the CPU, avoiding contention

Question 37

A small research team of three data scientists needs a dedicated deep-learning system that sits under a desk in their office, runs on standard single-phase wall power, and lets them prototype and fine-tune mid-sized models locally without requiring a data center or specialized rack cooling. They want an NVIDIA-branded turnkey system rather than assembling their own. Which option best fits these requirements?

Accepted Answer

An NVIDIA DGX Station-class workstation designed for office/deskside use

Answer

An NVIDIA DGX SuperPOD reference architecture deployment

Answer

A multi-node HGX cluster installed in a colocation data center

Answer

A rack of DGX H100 servers connected with InfiniBand

Question 38

A research lab is expanding its on-premises cluster from a single 8-GPU server to a multi-node system and wants to run a single large training job that spans 32 GPUs across four servers. During planning, an engineer asks how the collective communication (e.g., all-reduce during gradient synchronization) will behave across this configuration. Which statement correctly describes the interconnect roles in this scale-out design?

Accepted Answer

NVLink/NVSwitch provides high-bandwidth GPU-to-GPU communication within each server, while a high-speed fabric such as InfiniBand carries traffic between the four servers.

Answer

InfiniBand handles GPU-to-GPU communication inside each server, while NVLink connects the separate servers together across the rack.

Answer

PCIe alone is sufficient for both intra-server and inter-server GPU communication, so no specialized fabric is required.

Answer

NVLink extends natively across all four servers, so a separate inter-node network fabric is unnecessary for the 32-GPU job.

Question 39

An AI infrastructure team is designing the compute fabric for a multi-node DGX H100 SuperPOD cluster. They want to maximize the performance of distributed training collective operations (like all-reduce) across many nodes while ensuring each GPU has a dedicated, predictable path into the InfiniBand fabric. Which network design approach should they adopt for the compute fabric?

Accepted Answer

A rail-optimized InfiniBand topology where each GPU's dedicated NIC connects to a corresponding leaf switch rail across all nodes

Answer

A single shared 100GbE Ethernet switch aggregating all node traffic through one uplink

Answer

Rely solely on intra-node NVLink/NVSwitch and connect nodes with a single management-network cable

Answer

A converged fabric that routes both storage and compute traffic through the same oversubscribed spine to reduce switch count

Question 40

A research organization is building an AI cluster of 32 DGX H100 nodes for large-scale distributed training. The network architect must design the compute fabric so that inter-node GPU-to-GPU collective operations (like all-reduce during gradient synchronization) are not the bottleneck. Which fabric design best meets this requirement?

Accepted Answer

A dedicated non-blocking InfiniBand compute fabric with a rail-optimized topology, providing a separate high-speed link per GPU across nodes

Answer

A single shared 25 GbE Ethernet network carrying compute, storage, and management traffic together

Answer

PCIe cabling between the nodes to extend the intra-node GPU bus across the cluster

Answer

Relying solely on NVLink and NVSwitch to connect all 32 nodes into one memory domain

Question 41

A research organization is designing an on-premises AI cluster that must start with 40 GPUs but scale to over 500 GPUs within two years without redesigning the network fabric or storage architecture each time capacity is added. They want a validated, modular building-block approach that NVIDIA supports for predictable expansion. Which infrastructure design principle best fits this requirement?

Accepted Answer

Adopt a DGX SuperPOD reference architecture built from repeatable scalable units (SUs), so compute, networking, and storage expand as pre-validated increments

Answer

Deploy DGX systems as independent islands, each with its own isolated switch, and interconnect them ad hoc with Ethernet as new nodes are purchased

Answer

Buy the largest single HGX chassis available today and vertically scale by upgrading GPUs inside that one chassis over time

Answer

Start with a workstation-class DGX Station and later migrate the workload to a public cloud region when GPU demand grows

Question 42

An AI infrastructure team is deploying a multi-rack GPU cluster for large-model training. They need shared, high-throughput access to a multi-petabyte dataset from every compute node while ensuring dataset reads never compete for bandwidth with the GPU-to-GPU gradient synchronization traffic. Which infrastructure design best meets this requirement?

Accepted Answer

Provision a dedicated high-speed storage fabric with a parallel file system, kept physically separate from the compute (east-west) fabric used for inter-GPU communication

Answer

Attach local NVMe drives to each node and copy the full dataset to every server before each training run

Answer

Route both storage traffic and inter-GPU collective traffic over the same InfiniBand compute fabric to simplify cabling

Answer

Store the dataset in a single node's memory and stream it to all other nodes over the management Ethernet network

Question 43

A research team is provisioning storage for an on-premises NVIDIA DGX SuperPOD used to train large foundation models across dozens of nodes. They need a solution that can sustain the very high aggregate read/write throughput required to keep the GPUs fed with training data and to write large multi-terabyte checkpoints quickly without stalling the compute cluster. Which storage architecture best fits this requirement?

Accepted Answer

A high-performance parallel file system (e.g., NFS over RDMA or a certified parallel/scale-out filesystem) tuned to deliver the required aggregate bandwidth to all nodes

Answer

A single local SATA SSD installed in one node and shared to the others over a standard 1 GbE management network

Answer

Consumer-grade network-attached storage (NAS) connected via a single 1 GbE uplink

Answer

A tape-based archive tier accessed directly during each training step

Question 44

A junior analyst on your team is confused about terminology after a project kickoff. The team is building a system that automatically classifies medical images using a multi-layer neural network that learns features directly from raw pixel data without manual feature engineering. The analyst asks how this technique relates to the broader fields of artificial intelligence (AI) and machine learning (ML). Which statement correctly describes the relationship?

Accepted Answer

Deep learning is a subset of machine learning, which is itself a subset of artificial intelligence.

Answer

Machine learning is a subset of deep learning, which is a broader field than artificial intelligence.

Answer

Artificial intelligence, machine learning, and deep learning are three independent fields with no overlap.

Answer

Artificial intelligence is a subset of deep learning that only applies to neural networks with many layers.

Question 45

A cloud provider is building a multi-tenant GPU cluster where each host runs several isolated customer VMs. Operators notice that a significant fraction of host CPU cycles are consumed by virtual switching, encryption, and storage networking, reducing the CPU capacity available for tenant workloads. They want to offload this infrastructure processing off the host CPU while also enforcing per-tenant network isolation at line rate. Which component should they add to each server to address this?

Accepted Answer

An NVIDIA BlueField DPU to offload networking, storage, and security services from the host CPU

Answer

Additional CPU sockets to increase the number of cores available for infrastructure tasks

Answer

An NVSwitch to raise the all-to-all bandwidth between the GPUs in the server

Answer

A higher-capacity HBM3 GPU to accelerate the encryption workload on the accelerators

Question 46

A team is deploying a multi-node GPU training cluster in a public cloud region that does not offer InfiniBand, but does offer instances with RDMA over Converged Ethernet (RoCE) support at 400 Gb/s per node. The training job is a large distributed model that relies heavily on frequent all-reduce collective operations across nodes. Which statement best describes the appropriate networking approach for this deployment?

Accepted Answer

Use RoCE-enabled instances so RDMA can bypass the CPU/OS network stack and provide low-latency, high-bandwidth collectives comparable in approach to InfiniBand

Answer

Abandon the cloud deployment because distributed training collectives are impossible without native InfiniBand hardware

Answer

Use standard TCP/IP sockets over the Ethernet links, since RDMA offers no measurable benefit for GPU-to-GPU communication

Answer

Rely solely on intra-node NVLink and disable inter-node communication to avoid the slower Ethernet fabric

Question 47

A data science team already has a general-purpose large language model, but its answers on the company's internal insurance policies are often too generic. They gather thousands of labeled question-answer pairs specific to those policies and use them to further adjust the model's weights so it performs better on this domain. Which AI activity are they performing?

Accepted Answer

Fine-tuning, a form of training that updates model weights on domain-specific data

Answer

Inference, since the model is being used to answer policy questions

Answer

Quantization, since they are reducing the numerical precision of the model

Answer

Data labeling only, with no change to the model itself

Question 48

A media company wants to automatically produce first-draft marketing copy, generate original promotional images, and synthesize voice-overs for advertisements from short text prompts. The team asks which category of AI technology is best suited to power all three of these capabilities. Which technology should they adopt?

Accepted Answer

Generative AI models (e.g., large language models and diffusion models)

Answer

Classical regression models trained on historical sales figures

Answer

Rule-based expert systems using hand-written if-then logic

Answer

Clustering algorithms for unsupervised customer segmentation

Question 49

A media company wants to build an internal tool that produces original draft marketing copy, generates novel product imagery, and summarizes long documents into new prose. The data science lead asks which category of AI model is fundamentally suited to these tasks, as opposed to simply classifying or scoring existing inputs. Which type of model best matches the requirement?

Accepted Answer

Generative AI models, which learn the underlying data distribution to create new content such as text and images

Answer

Discriminative classification models, which assign predefined category labels to input samples

Answer

Linear regression models, which predict a continuous numeric value from input features

Answer

Clustering models, which group unlabeled data points by similarity without producing new samples

Question 50

An operations team notices that a rack of GPU servers is delivering lower-than-expected training throughput. Monitoring shows the GPUs are frequently reducing their clock speeds during sustained workloads, even though utilization stays near 100%. Inlet air temperatures at the top of the rack are significantly higher than at the bottom. What is the most likely root cause the team should address first?

Accepted Answer

Thermal throttling caused by inadequate cooling/airflow in the rack

Answer

Insufficient GPU memory capacity forcing frequent swapping to system RAM

Answer

A mismatched CUDA toolkit version in the container images

Answer

Network congestion on the InfiniBand fabric slowing gradient exchange

Question 51

A data science team is benchmarking a deep learning model training job. On a general-purpose server CPU with 32 high-performance cores, each iteration is slow, while the same job on a single GPU with thousands of smaller cores runs dramatically faster. A new team member asks why the GPU wins despite the CPU having faster individual cores. What is the best explanation of the architectural reason GPUs accelerate this workload?

Accepted Answer

The GPU has thousands of simpler cores designed to execute the same operation across many data elements in parallel, matching the massive matrix and vector math in neural network training.

Answer

The GPU runs each core at a much higher clock frequency than the CPU, so it completes sequential instructions faster.

Answer

The GPU offloads the operating system and I/O scheduling from the CPU, freeing the CPU to focus on the model math.

Answer

The GPU compresses the training data so fewer computations are required per iteration.

Question 52

A machine learning team currently trains a large model on a single 8-GPU node, and one full training epoch takes roughly 24 hours. Business stakeholders now require the same model to complete an epoch in about 3 hours to accelerate experimentation. Assuming the workload scales well across nodes and the interconnect is not the bottleneck, which infrastructure sizing decision best meets this target?

Accepted Answer

Scale out to approximately 8 nodes (64 GPUs) connected with high-speed InfiniBand for multi-node data-parallel training

Answer

Add more system RAM and faster CPUs to the existing single node while keeping the same 8 GPUs

Answer

Replace the local NVMe scratch drives with a larger object storage tier to hold more training data

Answer

Enable Multi-Instance GPU (MIG) to partition each of the 8 GPUs into smaller instances

Question 53

A research team runs a computational fluid dynamics simulation that requires high-precision double-precision (FP64) arithmetic. They plan to reuse an inference-optimized GPU that advertises very high FP8 and INT8 throughput but comparatively modest FP64 performance. Why might this GPU be a poor fit for their workload?

Accepted Answer

Inference-optimized GPUs prioritize low-precision throughput for AI, so their FP64 (double-precision) capability is limited and would bottleneck a scientific simulation that needs high numerical precision.

Answer

Inference GPUs lack CUDA cores entirely, so they cannot run any general-purpose numerical computation.

Answer

FP8 and INT8 formats provide more precision than FP64, so the simulation would produce inaccurate results due to excessive precision.

Answer

Scientific simulations can only run on CPUs, so no GPU regardless of precision support would help.

Question 54

A data science team benchmarks the same deep learning training job on a high-end multi-core CPU server and on a single data-center GPU. The GPU completes each training epoch roughly 30x faster. When asked to explain the primary architectural reason for this speedup to non-technical management, which explanation is most accurate?

Accepted Answer

The GPU contains thousands of smaller cores plus high-bandwidth memory, allowing massive parallel execution of the matrix and vector operations that dominate deep learning.

Answer

The GPU runs at a much higher clock frequency per core than the CPU, so each individual instruction executes far faster.

Answer

The GPU includes a larger operating-system cache that stores the entire dataset, eliminating disk reads during training.

Answer

The GPU offloads work to the CPU's idle cores, effectively doubling the number of sequential threads available for the job.

Question 55

A data science team is preparing to fine-tune a large language model on a single workstation. During their first attempt, the training job crashes immediately with an out-of-memory error before completing even one iteration. The model weights, optimizer states, and a modest batch of training data collectively exceed what can be held on the accelerator. The team wants to understand which GPU characteristic is the primary constraint they have hit, so they can plan a fix. Which hardware attribute is the limiting factor in this situation?

Accepted Answer

The total amount of GPU memory (VRAM) available to hold model weights, optimizer states, activations, and data

Answer

The GPU's memory bandwidth, which determines how fast data moves between memory and compute cores

Answer

The number of CUDA cores available for parallel floating-point computation

Answer

The clock speed of the GPU's streaming multiprocessors

Question 56

A research team is deploying a large language model whose weights are too large to fit within the memory of a single GPU. They want to run the model across four GPUs in one server, splitting the model's layers and parameters across the devices so that each GPU holds only a portion of the network. Which technique best describes this approach?

Accepted Answer

Model parallelism, where the model itself is partitioned across multiple GPUs

Answer

Data parallelism, where each GPU holds a full copy of the model and processes different batches of input

Answer

Gradient checkpointing, where activations are recomputed to reduce memory use

Answer

Mixed-precision training, where lower-precision formats reduce memory footprint

Question 57

A data center operations team runs several small inference services, each of which only uses a fraction of a single NVIDIA A100 GPU's compute and memory. They want to run multiple isolated workloads on one physical GPU simultaneously, with guaranteed, partitioned resources and fault isolation between tenants. Which NVIDIA GPU capability should they use?

Accepted Answer

Multi-Instance GPU (MIG), which partitions a single GPU into isolated instances with dedicated compute and memory

Answer

NVLink, which increases bandwidth for combining multiple GPUs into one large device

Answer

Tensor Cores, which accelerate mixed-precision matrix math within a workload

Answer

Unified Memory, which lets the GPU and CPU share a single address space

Question 58

A platform team runs a Kubernetes cluster with GPU worker nodes for AI inference workloads. They are struggling to keep NVIDIA drivers, the container runtime, device plugin, and monitoring components consistent across nodes, especially when adding new GPU nodes or upgrading driver versions. Which NVIDIA solution is specifically designed to automate the deployment and lifecycle management of these GPU software components on Kubernetes?

Accepted Answer

NVIDIA GPU Operator

Answer

NVIDIA Triton Inference Server

Answer

NVIDIA Base Command Manager

Answer

NVIDIA DCGM (Data Center GPU Manager) alone

Question 59

A data science team notices that their deep learning training jobs run dramatically faster on GPUs than on high-end CPUs. When explaining this to management, which characteristic of GPU architecture best accounts for the acceleration of neural network training workloads?

Accepted Answer

GPUs contain thousands of smaller cores that execute the same operation across many data elements in parallel, matching the matrix and vector math in neural networks

Answer

GPUs run at significantly higher clock speeds than CPUs, allowing each individual instruction to complete faster

Answer

GPUs use larger per-core cache hierarchies that eliminate the need to access system memory during training

Answer

GPUs process instructions strictly one at a time but with far lower latency per instruction than a CPU

Question 60

A data science team runs a recommendation model that must score millions of user records nightly. They currently process records one at a time on a CPU, and the batch job takes far too long. An infrastructure engineer suggests moving the workload to a GPU and processing records in large batches. Fundamentally, why does the GPU deliver a large speedup for this type of workload?

Accepted Answer

GPUs execute the same operation across thousands of data elements simultaneously using many parallel cores, making them ideal for high-throughput batched numerical work

Answer

GPUs run each individual record faster than a CPU because each GPU core has a higher clock speed than a CPU core

Answer

GPUs automatically compress the dataset so less data needs to be processed overall

Answer

GPUs eliminate the need for batching by processing records strictly one at a time with lower latency than a CPU

Question 61

A VDI administrator is consolidating several virtual desktop users onto a single data center GPU. Each user runs light graphics-accelerated design applications and requires flexible, time-shared access to the GPU rather than guaranteed hardware-isolated partitions. The team wants a scheduling-based virtualization approach that lets many VMs share one GPU efficiently. Which NVIDIA technology best fits this requirement?

Accepted Answer

NVIDIA vGPU software with time-sliced GPU scheduling

Answer

Multi-Instance GPU (MIG) with fixed hardware-isolated slices

Answer

NVLink bridging two GPUs into a single logical device

Answer

GPUDirect RDMA for direct network-to-GPU memory transfers

Question 62

A data center team is installing a rack of GPU servers, each drawing near its rated maximum during sustained training runs. Facilities reports that a single power feed to the rack occasionally experiences brief maintenance windows. The team wants training jobs to survive a loss of one power source without the servers shutting down. Which infrastructure design choice best meets this requirement?

Accepted Answer

Configure the servers with redundant (N+1) power supplies connected to two independent power feeds

Answer

Increase the airflow and cooling capacity in the rack to prevent thermal shutdown

Answer

Enable GPU clock throttling so each server draws less power during training

Answer

Add more NVLink connections between GPUs to distribute the compute load

Question 63

A data center operations team is planning to deploy several NVIDIA HGX H100 8-GPU systems in an existing on-premises facility. Each server draws roughly 10 kW at full load. The facility's current racks are provisioned for 7 kW per rack with traditional room-based air cooling. Before purchasing the systems, what infrastructure consideration should the team address first to ensure a viable deployment?

Accepted Answer

Evaluate the facility's power delivery and cooling capacity, since high-density GPU systems may exceed existing per-rack power and heat-removal limits and could require higher-capacity circuits and liquid or rear-door cooling

Answer

Order additional NVLink switches so the GPUs can communicate across racks at full bandwidth

Answer

Increase the object-storage capacity to hold model checkpoints before power is considered

Answer

Standardize all nodes on the same CUDA toolkit version to avoid container portability issues

Question 64

A media company needs to deploy a fleet of servers dedicated exclusively to real-time image generation inference for a customer-facing web service. Their priority is cost-effective, high-throughput inference with strong FP8/INT8 performance and good graphics/media engine capability, rather than large-scale distributed training. Which NVIDIA data center GPU is the most appropriate fit for this workload?

Accepted Answer

NVIDIA L40S, which balances inference throughput, FP8 support, and media/graphics acceleration at a lower cost point

Answer

NVIDIA H100 SXM configured in an 8-GPU NVLink domain for maximum training throughput

Answer

NVIDIA A100 80GB in a multi-node InfiniBand cluster optimized for FP64 workloads

Answer

NVIDIA GH200 Grace Hopper Superchip for coherent CPU-GPU memory in large-model training

Exam domain	Exam weight	Practice
Essential AI Knowledge	38%	Practice this topic
AI Infrastructure	40%	Practice this topic
AI Operations	22%	Practice this topic

NVIDIA-Certified Associate: AI Infrastructure and Operations (NCA-AIIO) practice exam & study guide

Start studying NCA-AIIO

All NCA-AIIO study resources

NCA-AIIO exam domains

Sample NCA-AIIO questions

Key NCA-AIIO terms

NCA-AIIO frequently asked questions

Essential AI Knowledge (38%)

AI Infrastructure (40%)

AI Operations (22%)