NVIDIA Certified Professional - AI Infrastructure (NCP-AII) Flashcards¶
43 cards from this guide's fact sheet and notes. Click a question to reveal the answer.
To study with spaced repetition, download flashcards.csv and import it into Anki or Quizlet (comma separated, first row is the header). Back to the study guide.
Exam logistics¶
How long is the NVIDIA Certified Professional - AI Infrastructure (NCP-AII) exam?
120 minutes
How many questions are on the NVIDIA Certified Professional - AI Infrastructure (NCP-AII) exam?
60-70 questions
What is the passing score for NVIDIA Certified Professional - AI Infrastructure (NCP-AII)?
Not officially published
What does the NVIDIA Certified Professional - AI Infrastructure (NCP-AII) exam cost?
$200 USD
How long is NVIDIA Certified Professional - AI Infrastructure (NCP-AII) valid for?
2 years
Dgx systems¶
Inference serving (dgx systems)
Multiple small models on one GPU
Multi-tenancy (dgx systems)
Isolated GPU resources per tenant
Development (dgx systems)
Share expensive GPUs among developers
CI/CD (dgx systems)
GPU testing pipelines with resource isolation
Gpu clusters¶
Prefetching (gpu clusters)
Load next batch while GPU processes current
Multiple data workers (gpu clusters)
Parallel data loading processes
Data format (gpu clusters)
Use efficient formats (TFRecord, WebDataset)
Caching (gpu clusters)
Cache preprocessed data on local NVMe
Compression (gpu clusters)
Reduce storage I/O with compressed datasets
Job scheduling¶
slurmctld (job scheduling)
Central controller daemon
slurmd (job scheduling)
Compute node daemon
slurmdbd (job scheduling)
Database daemon for accounting
sbatch (job scheduling)
Submit batch jobs
srun (job scheduling)
Run interactive/parallel jobs
sacct (job scheduling)
View job accounting
Low GPU utilization (job scheduling)
Increase batch size, check data loading
GPU memory underutilized (job scheduling)
Increase batch size or model size
CPU bottleneck (job scheduling)
More data workers, preprocess data offline
Network bottleneck (job scheduling)
Optimize collective operations, check NCCL config
I/O bottleneck (job scheduling)
Stage data to local NVMe, use parallel FS
Performance tuning¶
GPU Utilization (performance tuning)
Percentage of time SMs are active
Memory Utilization (performance tuning)
Percentage of GPU memory in use
Power Draw (performance tuning)
Current power consumption in watts
Temperature (performance tuning)
GPU die temperature in Celsius
Clock Speeds (performance tuning)
Current SM and memory clock frequencies
ECC Errors (performance tuning)
Single-bit (correctable) and double-bit (uncorrectable) errors
NVLink Throughput (performance tuning)
Bandwidth utilization per NVLink connection
PCIe Throughput (performance tuning)
Bandwidth utilization on PCIe bus
Prefetching (performance tuning)
Overlap data loading with GPU computation
Multi-worker loading (performance tuning)
Parallel data reader processes
Data format (performance tuning)
Use efficient formats (WebDataset, LMDB, TFRecord)
Local caching (performance tuning)
Stage hot data on NVMe SSDs
GPUDirect Storage (performance tuning)
Bypass CPU for storage-to-GPU path
MLPerf Training (performance tuning)
Industry standard for training performance
MLPerf Inference (performance tuning)
Industry standard for inference throughput
nccl-tests (performance tuning)
Network collective operation benchmarks
FIO (performance tuning)
Storage I/O benchmarks
IOR (performance tuning)
Parallel file system benchmarks