NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL) Flashcards¶

129 cards from this guide's fact sheet and notes. Click a question to reveal the answer.

To study with spaced repetition, download flashcards.csv and import it into Anki or Quizlet (comma separated, first row is the header). Back to the study guide.

Exam logistics¶

How long is the NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL) exam?

60 minutes

How many questions are on the NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL) exam?

50-60 multiple choice questions

What is the passing score for NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL)?

70% (scaled scoring)

What does the NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL) exam cost?

$125 USD

How long is NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL) valid for?

2 years

How is the NVIDIA Certified Associate - Generative AI and LLMs (NCA-GENL) exam delivered?

Online proctored

Transformer architecture¶

Residual connection (transformer architecture)

input is added to sub-layer output (skip connections)

Layer normalization (transformer architecture)

normalizes activations for training stability

Vocabulary size trade-offs (transformer architecture)

Larger vocab = more direct token mappings but larger embedding table

Token efficiency (transformer architecture)

English text typically tokenizes at ~1.3 tokens per word

Multilingual (transformer architecture)

Non-English languages often need more tokens per concept

Context window (transformer architecture)

Token count determines effective input length

Special tokens (transformer architecture)

BOS (beginning), EOS (end), PAD (padding), UNK (unknown)

Chinchilla scaling (transformer architecture)

Optimal training uses ~20 tokens per parameter

Nemotron (transformer architecture)

NVIDIA's family of LLMs

Text generation/chat (transformer architecture)

Decoder-only (GPT, LLaMA)

Classification/understanding (transformer architecture)

Encoder-only (BERT) or decoder-only with fine-tuning

Translation/summarization (transformer architecture)

Encoder-decoder (T5) or decoder-only

Embeddings (transformer architecture)

Encoder-only or specialized embedding models

Prompt engineering¶

Task accuracy (prompt engineering)

correctness of outputs

Format compliance (prompt engineering)

adherence to specified output format

Consistency (prompt engineering)

same input produces similar outputs

Safety (prompt engineering)

absence of harmful content

Relevance (prompt engineering)

output addresses the input question/task

Factual extraction (prompt engineering)

Zero-shot + low temperature + structured output

Classification (prompt engineering)

Few-shot with representative examples

Complex reasoning (prompt engineering)

Chain-of-thought with step-by-step

Creative generation (prompt engineering)

Higher temperature + top-p sampling

Code generation (prompt engineering)

Low temperature + few-shot with code examples

Rag vector databases¶

Reduces hallucination (rag vector databases)

grounds responses in retrieved evidence

Dynamic knowledge (rag vector databases)

external data can be updated without retraining

Source attribution (rag vector databases)

can cite specific documents for transparency

Domain adaptation (rag vector databases)

adds domain-specific knowledge without fine-tuning

Cost-effective (rag vector databases)

cheaper than retraining for knowledge updates

Privacy (rag vector databases)

keeps sensitive data in a controlled retrieval system

Small chunks (128-256 tokens) (rag vector databases)

More precise retrieval, less context per chunk

Medium chunks (256-512 tokens) (rag vector databases)

Good balance for most use cases

Large chunks (512-1024 tokens) (rag vector databases)

More context, but may include irrelevant info

Overlap (rag vector databases)

Prevents information loss at chunk boundaries

NVIDIA NV-Embed (rag vector databases)

NVIDIA's embedding models, available via NIM

E5-large-v2 (rag vector databases)

Strong open-source embedding model

BGE (BAAI General Embeddings) (rag vector databases)

High-quality multilingual embeddings

GTE (General Text Embeddings) (rag vector databases)

Alibaba's embedding models

Cosine similarity (rag vector databases)

measures angle between vectors (most common)

Dot product (rag vector databases)

measures magnitude-weighted similarity

Euclidean distance (L2) (rag vector databases)

measures geometric distance

Inner product (rag vector databases)

similar to dot product, used in some implementations

Pinecone (rag vector databases)

fully managed, serverless vector database

Weaviate (rag vector databases)

open-source with hybrid search (vector + keyword)

Chroma (rag vector databases)

lightweight, developer-friendly, good for prototyping

Qdrant (rag vector databases)

high-performance with advanced filtering

pgvector (rag vector databases)

PostgreSQL extension for vector search

Query expansion (rag vector databases)

rephrase or expand the user query

HyDE (Hypothetical Document Embeddings) (rag vector databases)

generate a hypothetical answer, use it for retrieval

Multi-query (rag vector databases)

generate multiple query variations, combine results

Step-back prompting (rag vector databases)

generate a broader query for complex questions

Parent-child chunking (rag vector databases)

retrieve small chunks, return parent (larger) chunks for context

Sentence window retrieval (rag vector databases)

retrieve at sentence level, expand to surrounding sentences

Knowledge graph integration (rag vector databases)

combine vector search with graph traversal

Multi-hop retrieval (rag vector databases)

iterative retrieval for complex multi-step questions

Recall@k (rag vector databases)

fraction of relevant documents in top-k results

Precision@k (rag vector databases)

fraction of top-k results that are relevant

MRR (Mean Reciprocal Rank) (rag vector databases)

average of 1/rank of first relevant result

NDCG (rag vector databases)

normalized discounted cumulative gain

Faithfulness (rag vector databases)

is the answer supported by retrieved context?

Answer relevancy (rag vector databases)

does the answer address the question?

Context relevancy (rag vector databases)

are retrieved documents relevant to the question?

Fine tuning customization¶

Model weights (fine tuning customization)

2 bytes per parameter (FP16)

Gradients (fine tuning customization)

2 bytes per parameter

Optimizer states (fine tuning customization)

8 bytes per parameter (Adam: momentum + variance, FP32)

Activations (fine tuning customization)

varies with batch size and sequence length

Catastrophic forgetting (fine tuning customization)

model loses pre-trained knowledge

Overfitting (fine tuning customization)

too few examples relative to parameters

Expensive (fine tuning customization)

requires significant GPU compute

Not composable (fine tuning customization)

cannot easily switch between tasks

Rank (r) (fine tuning customization)

Dimensionality of low-rank matrices

Alpha (fine tuning customization)

Scaling factor (typically alpha = 2 * rank)

Target modules (fine tuning customization)

Which layers to apply LoRA to

Dropout (fine tuning customization)

Regularization for LoRA layers (0.05-0.1 typical)

NF4 (Normal Float 4-bit) (fine tuning customization)

Information-theoretically optimal 4-bit data type

Double quantization (fine tuning customization)

Quantize the quantization constants for additional savings

Paged optimizers (fine tuning customization)

Use CPU memory for optimizer state overflow

Diversity (fine tuning customization)

cover the range of expected use cases

Accuracy (fine tuning customization)

ensure labels and responses are correct

Consistency (fine tuning customization)

maintain uniform formatting and style

Balance (fine tuning customization)

represent all categories/tasks proportionally

Relevance (fine tuning customization)

match the target deployment scenario

LoRA fine-tuning (fine tuning customization)

1,000-10,000 examples (quality over quantity)

Full fine-tuning (fine tuning customization)

10,000-100,000+ examples

RLHF/DPO (fine tuning customization)

5,000-50,000 preference pairs

SFT alignment (fine tuning customization)

10,000-50,000 instruction-response pairs

Perplexity (fine tuning customization)

how well the model predicts held-out text (lower is better)

BLEU (fine tuning customization)

n-gram overlap with reference (translation, summarization)

ROUGE (fine tuning customization)

recall-based overlap with reference (summarization)

F1 Score (fine tuning customization)

balanced precision/recall for classification tasks

Exact Match (fine tuning customization)

percentage of exactly correct answers

Helpfulness (fine tuning customization)

does the response address the question?

Harmlessness (fine tuning customization)

is the response safe and appropriate?

Coherence (fine tuning customization)

is the response well-organized and clear?

Preference ranking (fine tuning customization)

side-by-side comparison of model outputs

Deployment inference¶

Pre-optimized containers (deployment inference)

ready-to-deploy Docker images

OpenAI-compatible API (deployment inference)

familiar chat completion and completion endpoints

Automatic optimization (deployment inference)

selects best configuration for the available GPU

Multiple model support (deployment inference)

LLMs, embedding models, reranking models

Health monitoring (deployment inference)

built-in health check endpoints

Enterprise support (deployment inference)

part of NVIDIA AI Enterprise platform

Multi-framework support (deployment inference)

TensorRT, PyTorch, TensorFlow, ONNX, Python

Dynamic batching (deployment inference)

groups incoming requests for efficiency

Model ensembles (deployment inference)

chain multiple models in a pipeline

Concurrent model execution (deployment inference)

run multiple models on same GPU

Model versioning (deployment inference)

A/B testing and rolling updates

Metrics (deployment inference)

Prometheus-compatible monitoring

gRPC and HTTP (deployment inference)

dual protocol support

Easy deployment (deployment inference)

NIM (Docker container, OpenAI API)

Custom optimization (deployment inference)

TensorRT-LLM (compile and optimize)

Multi-model serving (deployment inference)

Triton Inference Server

Maximum throughput (deployment inference)

Continuous batching + quantization

Ethics responsible ai¶

Stereotyping (ethics responsible ai)

associating groups with fixed characteristics

Representation bias (ethics responsible ai)

over/under-representing certain groups

Toxicity (ethics responsible ai)

generating harmful, offensive, or inappropriate content

Sentiment bias (ethics responsible ai)

different sentiment when discussing different groups

Allocation bias (ethics responsible ai)

different quality of service for different users

Harmful content (ethics responsible ai)

violence, self-harm, illegal activities

Bias and discrimination (ethics responsible ai)

stereotypes, unfair treatment

Privacy violations (ethics responsible ai)

PII extraction, personal information disclosure

Misinformation (ethics responsible ai)

factually incorrect claims, conspiracy theories

Jailbreaking (ethics responsible ai)

bypassing system prompts and safety measures

Prompt injection (ethics responsible ai)

overriding model instructions

Copyright (ethics responsible ai)

reproducing copyrighted material