DCGM
NVIDIA Data Center GPU Manager (DCGM) is a suite of tools for monitoring and managing GPU health and utilization in a cluster. NCA-AIIO covers it for monitoring in the AI operations domain.
All NCA-AIIO Terms
GPU
A GPU (graphics processing unit) is a massively parallel processor that accelerates AI training and inference far beyond a CPU.
CUDA
CUDA is NVIDIA's parallel computing platform and programming model that lets software use the GPU for general-purpose computation.
cuDNN
cuDNN (CUDA Deep Neural Network library) is an NVIDIA library of GPU-accelerated primitives for deep learning frameworks.
Tensor Core
A Tensor Core is a specialized unit in NVIDIA GPUs that accelerates the matrix math used in deep learning training and inference.
NVIDIA DGX
NVIDIA DGX is a family of purpose-built AI systems that integrate multiple data center GPUs, networking, and software for training and inference.
NVLink
NVLink is NVIDIA's high-speed interconnect that links GPUs directly for fast memory sharing within and across systems.
NVSwitch
NVSwitch is an NVIDIA switch chip that connects many GPUs over NVLink so they communicate at full bandwidth.
InfiniBand
InfiniBand is a high-throughput, low-latency networking fabric used to connect nodes in AI and HPC clusters.
Triton Inference Server
NVIDIA Triton Inference Server is open-source software that serves trained AI models at scale across frameworks and hardware.
NVIDIA AI Enterprise
NVIDIA AI Enterprise is a supported software suite of frameworks and tools for developing and deploying AI on NVIDIA infrastructure.
NGC
NGC (NVIDIA GPU Cloud) is NVIDIA's catalog of GPU-optimized containers, models, and Helm charts for AI workloads.
Multi-Instance GPU
Multi-Instance GPU (MIG) is a capability that partitions a single NVIDIA GPU into isolated instances for multi-tenant workloads.
DCGM
NVIDIA Data Center GPU Manager (DCGM) is a suite of tools for monitoring and managing GPU health and utilization in a cluster.
Inference
Inference is the phase where a trained AI model generates predictions on new data, as opposed to training.