AWS Certified Generative AI Developer - Professional (AIP-C01) Flashcards¶

205 minutes

85 total (65 scored + 10 unscored)

750 / 1000 (scaled)

$300 USD (50% discount voucher available for prior cert holders)

3 years

Pearson VUE (online proctored or testing center)

org-level adoption

per-workload review

API-only access to multiple model providers (Anthropic, AI21, Cohere, Meta, Stability, Mistral, Amazon Titan)

change model ID, re-run, compare

every model artifact gets a version in the Registry

status gates in the Registry; aws sagemaker update-model-package flips status

CodePipeline / Step Functions watches Registry, triggers SageMaker Endpoint update on approval

keep previous endpoint config; update-endpoint to roll back

schedule cutover, monitor blue/green, decommission old endpoints to control cost

keep system prompt in system, alternate user / assistant turns, never repeat user twice in a row (most providers reject).

more shards = more parallelism but more overhead. Common rule: 30-50 GB per shard.

separate index per domain (legal, support, products) to improve precision and reduce noise.

route queries to a coarse index first, then drill into a fine index based on classification.

HNSW (default in OpenSearch k-NN) is the standard; tune m, ef_construction, ef_search for recall/latency trade-off.

Bedrock Knowledge Bases supports incremental sync (only re-embed changed objects). Trigger via API or schedule.

S3 event notifications → EventBridge → Lambda → Knowledge Base ingestion API.

EventBridge Scheduler or Step Functions cron runs full re-sync nightly/weekly.

test on a held-out eval set; don't assume the highest-priced is best.

good for semantic similarity, can miss exact keyword matches.

run both, combine scores. OpenSearch supports natively. Bedrock Knowledge Bases offers HYBRID search type. Almost always beats pure vector for production.

generate synonyms / paraphrases with a small Bedrock model, run multiple retrievals, merge.

break a multi-part question into sub-questions (Lambda invokes Bedrock to decompose), retrieve per sub-question, fuse.

rewrite a conversational follow-up ("when was that?") into a standalone query ("when was the iPhone 15 released?") using prior chat history.

define a vector_search tool in your FM tool schema. The FM decides when to call it. Standard for Anthropic Claude tool use, Cohere Command-R, Llama 3.1+, Nova.

open standard for connecting FMs to external tools. AWS supports MCP servers running on Lambda (stateless tools) or ECS (stateful/complex tools). MCP clients in your agent let it discover and call tools without bespoke wiring.

define a single retrieval API (e.g., POST /retrieve with query, filter, top_k) that wraps Bedrock KB / OpenSearch / Kendra. Clients don't care which backend.

Lambda runs the prompt against a fixture set, checks expected output (regex / JSON schema validation).

Step Functions iterates through difficult inputs (long contexts, adversarial inputs, non-English) and reports pass/fail.

CloudWatch metric on test pass rate per prompt version; alarm on regressions.

separate fields with XML tags (, , ) so the model can attend to them clearly.

require JSON conforming to a schema; validate post-hoc; reject and retry on schema violation.

ask the model to reason step-by-step before final answer. Improves quality on multi-step problems but increases token usage.

have the model generate multiple answers and pick the most consistent, or critique its own answer and revise.

collect user rating + final outcome; periodically re-evaluate prompts against this dataset.

prompt-centric workflows; non-developer authors; built-in Bedrock integration

developer-built; needs full AWS service access, complex error handling, long-running execution

parameterized templates, versions, approval workflows.

orthogonal layer (content filters, denied topics, contextual grounding, sensitive info, word filters).

no-code/visual prompt orchestration; pick over Step Functions when prompt-centric and non-dev authors.

conversation history in the prompt itself. Bounded by context window.

store conversation history + user state in DynamoDB (low-latency reads), retrieve relevant slices, inject into prompt.

managed memory for agents (semantic + episodic) so you don't reinvent it.

a coordinator agent dispatches sub-tasks to specialist agents (e.g., research agent, coding agent, summarizer). Combine with AWS Agent Squad.

agents in sequence, each handing off (e.g., extract → classify → respond).

multiple agents independently solve the same problem; aggregate.

route easy queries to a cheap model; escalate hard ones. E.g., Haiku → Sonnet → Opus only on retry. Use API Gateway / Step Functions / Lambda for routing.

classification, extraction, intent detection often work great with small models or even custom-trained encoders on SageMaker.

enterprise IdP → IAM Identity Center (formerly AWS SSO) → app roles. Federate ML/FM access without managing IAM users.

IAM policies on Bedrock InvokeModel / Knowledge Base actions, scoped per role. Tag-based ABAC for tenant-level access.

each Lambda / agent role can only invoke specific models, specific Knowledge Bases, specific Guardrails.

rack of AWS infrastructure on-prem. Gives you SageMaker AI / EC2 with same APIs locally - useful when data can't leave premises.

AWS infra inside telecom carrier networks for low-latency 5G edge.

out of scope per official guide.

trigger logic at CloudFront edge for low-latency.

compile models for edge devices (then deploy to non-AWS edge).

visual workflow builder for prompt chains - good for non-developers building automations.

build internal GenAI apps over your enterprise data with no code (knowledge base + actions).

visual editor for state machines (developer-oriented).

Java upgrades, .NET porting

Bedrock Agents (managed) → Bedrock AgentCore (production runtime) → Strands Agents (code SDK) → AWS Agent Squad (multi-agent) → Step Functions (custom orchestration).

AWS-native tool servers; Lambda for stateless, ECS for stateful, MCP clients in agents.

CoT, ReAct, plan-and-execute, reflection, tree of thoughts.

short (in-context), long (DynamoDB), episodic (vector store), AgentCore Memory (managed).

Step Functions stopping conditions, Lambda timeouts, IAM boundaries, circuit breakers, Guardrails, output validation.

supervisor + workers (AWS Agent Squad), pipelines, ensembles.

A2I (review), Ground Truth (labeling), Step Functions wait-for-callback (approval), API Gateway (feedback).

Bedrock on-demand vs Provisioned Throughput vs SageMaker endpoints (real-time / async / serverless / batch / MME).

Model cascading: cheap → expensive escalation; reduces cost.

API Gateway (sync), EventBridge (async), AppFlow (SaaS), DataSync (file), DMS (CDC), VPC endpoints (private).

IAM Identity Center for federation; IAM RBAC; tag-based ABAC; least privilege.

Outposts (on-prem), Wavelength (5G edge), Lambda@Edge.

CodePipeline + CodeBuild + automated tests + Bedrock Model Evaluations + GenAI gateway deploy.

SDK backoff, API Gateway rate limit, circuit breakers, fallback chain, X-Ray traces.

static (AppConfig), content-based (Step Functions Choice), metric-based (Lambda+CloudWatch), provider-aware (API Gateway transforms).

Amplify, API Gateway, Cognito.

Bedrock Prompt Flows, Q Business Apps.

Lambda+webhook for CRM, Step Functions for doc workflows, Q Business for internal knowledge, Bedrock Data Automation for end-to-end doc understanding.

CloudWatch Logs Insights for prompt analysis, X-Ray for traces, Q Developer for error pattern recognition.

dedicated check for response groundedness against provided context. Configurable threshold (e.g., 0.75 = require 75%+ groundedness score).

retrieve facts, model uses them, Guardrails verifies grounding.

request the model output a confidence score; reject responses below threshold; route low-confidence to human review (A2I).

compare model claims against retrieved evidence with embedding similarity.

require model to output JSON conforming to a schema; deterministic parse + validate; reject malformed.

when the question is data lookup, use the FM to generate SQL, run it deterministically against the DB; the answer becomes facts, not hallucinations.

Lambda execution role can invoke only specific Bedrock model ARNs / Knowledge Base ARNs / Guardrail ARNs.

bucket policies restricting which IAM principals can read training data / vector store backups.

tag models / KBs / agents with tenant=acme, IAM policy condition aws:ResourceTag/tenant = ${aws:PrincipalTag/tenant} so each tenant can only access its own resources.

detect unintended cross-account / public access on Bedrock / SageMaker / S3 resources.

federate enterprise IdP for human access; permission sets per role.

row-level + column-level + cell-level access control on data lake assets feeding RAG / training. Grant specific columns of a table to specific principals.

replace PII with reversible tokens stored in DynamoDB / KMS; FM sees tokens, app de-tokenizes for the user

tier to S3 IA / Glacier / delete after N days. Enforce regulatory retention windows.

who called what API (control plane). Bedrock has data events for InvokeModel - enable to capture per-invocation audit trails.

per-call prompt + response payloads delivered to S3 and/or CloudWatch Logs. Configurable in Bedrock account settings.

ISO/IEC 42001 (AI Management System)

AWS Cloud Adoption Framework for AI/ML/GenAI (CAF-AI)

AWS Generative AI Lifecycle Operational Excellence framework

log per-response confidence score; surface to users

Knowledge Base RetrieveAndGenerate returns citations; show them

log thought / action / observation steps; surface to internal users for debugging

external documentation of capabilities and limits

AWS handles capacity; you handle throttling with backoff. Use Cross-Region Inference for headroom.

explicitly buy model units. Estimate from peak_tpm = peak_rps * avg_tokens_per_request.

instance count + auto scaling target tracking on InvocationsPerInstance or custom metric.

factor that into the customization decision.

parallelize independent FM calls; respect Provisioned Throughput TPS limits

shorten outputs (maxTokens); use stop sequences to terminate early

predictable TPS without throttling

CloudWatch metric TokensIn per minute, anomaly detector → alarm. Catches runaway loops.

hash-based or semantic similarity drift in outputs over time. Custom Lambda computes drift, publishes to CloudWatch, alarm on breach.

AWS Cost Anomaly Detection at the service or tag level (e.g., per Bedrock usage tag). Auto-detection + email / SNS / EventBridge.

rate of failed tool calls in an agent; track in CloudWatch; alarm on regression.

each agent emits CloudWatch metrics; correlation IDs in logs to trace cross-agent flows

OpenSearch / Aurora metrics on query latency, recall, replication lag

scheduled Lambda to compact / reindex; OpenSearch auto-merges

post-ingestion spot-check against fixtures; CloudWatch metrics on ingestion failures

scheduled refresh + monitoring of last-sync timestamp

a curated set of inputs with known correct outputs. Run periodically, score outputs, alarm on regression. Detects hallucinations and quality drift.

compare current model output against historical outputs for the same input. Alert on semantic drift. Useful when you can't define ground truth but want consistency.

log model thought processes (CoT outputs, Bedrock Agent traces) and analyze for logical errors patterns.

CloudWatch Logs Insights queries built specifically for prompt analysis (token distribution, response time vs prompt length, error pattern matching).

response addresses the question

claims match ground truth

same input → same / similar output across runs

grammatical, readable

actually solves the user's problem

matches brand voice / register

no harmful content

equal performance across demographic slices

test one model on your data.

run two models, compare metrics or human/judge preferences.

specifically for Knowledge Bases - retrieval quality + generation quality.

deploy new prompt / model to a small fraction of users; compare metrics (latency, cost, quality, error rate)

avg input tokens, avg output tokens per request

quality score / p95 latency

dollar amount / completed-task count

deflection rate, conversion, NPS

fraction of relevant docs in top-K

fraction of top-K that are relevant

1 / rank of first relevant

rank-aware relevance

measure of retrieved content's semantic match to query

p50, p95 of retrieval call

answer supported by context

addresses the question

cited sources actually appear in retrieved context

built-in eval jobs for retrieval and generation

what % reach successful final answer

right tool for the step

well-formed args

LLM-as-judge on agent traces

efficiency proxy

Lambda + EventBridge Scheduler emails dashboards as PDFs / markdown to stakeholders

golden dataset runs scoring hallucination rate + semantic drift

bundle related actions backed by Lambda functions or OpenAPI-defined APIs. The agent's "tools."

attach a Bedrock Knowledge Base for RAG within the agent.

apply input/output safety checks on every model invocation in the agent loop.

customize the system prompt for each agent stage (pre-processing, orchestration, post-processing).

similar to Lambda. Versions are immutable; aliases (PROD, STAGING) point at versions.

structured trace of agent reasoning, tool calls, observations - critical for debugging and exam scenarios.

per-session memory passed across turns.

instead of Bedrock invoking your action Lambda directly, hand control back to the caller to execute the action (useful for client-side actions or async ops).

secure, isolated execution of agents

managed agent memory (semantic + episodic)

per-agent identity for fine-grained access control

bridges agents to enterprise tools (MCP, OpenAPI, etc.)

sandboxed browser tool for agents to use the web

sandboxed code execution tool

traces, metrics, prompts/responses logged

a router agent dispatches to specialist agents

agents in series

Persistent state across agents

exposes a set of tools (and resources, prompts)

lives in the agent runtime, connects to one or more MCP servers, presents tools to the FM

per-session key-value state

state passed through state machine

what % of agent runs reach a successful final answer

is the agent picking the right tool for the task

are the arguments well-formed

manual or LLM-as-judge review of trace

observed at API Gateway / Lambda

automated evaluation of agent traces

Managed access to foundation models from multiple providers (Anthropic, AI21, Cohere, Meta, Mistral, Stability, Amazon Titan/Nova). The central service for this exam.

model invocation (InvokeModel, Converse), streaming (ConverseStream), batch inference, fine-tuning, continued pre-training, Provisioned Throughput, Cross-Region Inference, prompt caching.

Domains 1, 2, 3, 4, 5 (all).

DeepRacer, DeepComposer, DevOps Guru, Forecast, Fraud Detector, Lookout for Equipment / Metrics / Vision, HealthLake, Monitron, Panorama

Batch, Beanstalk, Lightsail, Snow Family

Redshift, Timestream, Keyspaces, QLDB

SES, IoT family, GameLift, Braket, all media services, Direct Connect, Transit Gateway

default, fixed-size, hierarchical, semantic, no chunking

Titan v2, Titan Multimodal, Cohere Embed

OpenSearch Serverless (default), Aurora pgvector, MongoDB Atlas, Pinecone, Redis Enterprise, Neptune Analytics

returns top-K chunks for a query

retrieve + invoke FM with retrieved context, returns response with citations

Lambda functions or OpenAPI-defined APIs as the agent's tools

Trace of reasoning + tool calls + observations

secure isolated execution

managed semantic + episodic memory

per-agent IAM-style identity for fine-grained access

bridges to enterprise tools (MCP, OpenAPI)

sandboxed browser tool

sandboxed code execution

tracing, metrics, logs

immutable versions

status gates before promotion

output of one prompt feeds into the next

invoke Lambda or AWS service calls in-flow

in-flow retrieval

prompt-centric workflows, non-developer authors, built-in Bedrock integration, simpler authoring

developer-built, full AWS service access, complex error handling, long-running

managed RAG; connectors (S3, Web Crawler, Confluence, SharePoint, Salesforce, custom); chunking; embeddings (Titan/Cohere); vector stores (OpenSearch Serverless default, Aurora pgvector, etc.); Retrieve and RetrieveAndGenerate APIs; metadata filters; HYBRID search; reranking.

action groups, KB assoc, Guardrails assoc, prompt overrides, versions/aliases, trace, session state, return of control.

Runtime, Memory, Identity, Gateway, Browser, Code Interpreter, Observability - production runtime.

parameterized templates, versions, aliases, approval workflows.

automatic, human, LLM-as-judge; single or multi-model; RAG-specific.

end-to-end PDF/image/audio/video → structured.

persistent instructions, role, constraints. NOT a turn; shapes all subsequent turns.

the input you want a response to.

prior model outputs (when continuing a conversation).

conventional wisdom is to fix one and adjust the other

simplest, may lose important context

have an FM summarize old turns into a few sentences; insert summary; drop verbatim turns

maintain a rolling summary that's updated each turn

drop old turns from prompt but keep them retrievable; retrieve relevant ones at each turn

different models, parameters, prompt versions in one logical prompt

immutable versions

output of one prompt feeds the next

route based on outputs (e.g., classifier → routing)

call existing templates

any AWS service call inline

in-flow retrieval

call a Bedrock Agent within the flow

read/write objects

loop over arrays

similar to Lambda

parameterized templates, versions, aliases, approval, reusable across Agents + Prompt Flows.

visual/no-code chains, conditional branching, KB nodes, Lambda nodes, Agent nodes; pick over Step Functions when prompt-centric and non-dev.

you can show which document supported the answer (citations).

1024-dim vectors take 4× the storage of 256-dim. For most enterprise corpora, 512 is a good sweet spot.

pick a multilingual model if any documents or queries are non-English.

embeddings trained on web text may underperform on legal, medical, code. If you have eval data, test 2-3 candidates.

measures direct distance.

faster on normalized vectors.

how many candidates to retrieve. 5-20 typical; larger for reranking pipelines.

searchType: HYBRID in retrieve API

Cohere Rerank (multiple versions)

dedicated check that the response is supported by provided context. Filters out ungrounded/hallucinated responses.

parse model JSON output, validate cited sources actually appear in retrieved chunks.

of all relevant docs, how many appear in the top-K retrievals

of the top-K retrievals, how many are relevant

1 / rank of the first relevant doc

relevance-weighted ranking quality

response is supported by retrieved context

response actually addresses the question

retrieved context actually relates to the question

against ground truth

built-in evaluation jobs that score retrieval and generation quality on a labeled eval set.

automated, human, and LLM-as-judge for generation quality.

bias detection and explainability.

call the database directly, don't go through an FM.

use code/Lambda, not RAG.

tool calling against an API beats RAG over user records.