Tag: Azure AI

AI, AI-103, Artificial Intelligence (AI), Azure AI, Microsoft Certification May 25, 2026

Govern agent behavior with oversight modes, constraints, and tool-access controls (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Implement responsible AI across generative AI and agentic systems
      --> Govern agent behavior with oversight modes, constraints, and tool-access controls

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

AI agents are becoming increasingly capable of:

Retrieving enterprise data
Executing tools
Calling APIs
Managing workflows
Performing multi-step reasoning
Making autonomous decisions

Unlike traditional AI chatbots, agentic systems can:

Interact with external systems
Trigger business actions
Access sensitive information
Operate semi-autonomously

Because of this, governance and oversight are critical.

Organizations must ensure agents behave safely, reliably, and within approved boundaries.

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of responsible AI governance for agent-based systems.

For the AI-103 exam, you should understand:

Agent governance principles
Oversight modes
Human-in-the-loop systems
Tool-access controls
Permission boundaries
Agent constraints
Approval workflows
Risk mitigation
Prompt injection prevention
Responsible AI principles
Agent security and compliance
Safe autonomous behavior

Why Agent Governance Matters

AI agents can create significant risks if poorly governed.

Examples include:

Unauthorized actions
Data leakage
Harmful outputs
Excessive automation
Unsafe tool execution
Prompt injection attacks
Compliance violations

Strong governance helps:

Reduce operational risk
Protect enterprise systems
Improve trust
Ensure compliance
Prevent misuse

What Is Agent Governance?

Agent governance refers to policies and controls that regulate:

Agent behavior
Decision-making
Tool usage
Data access
Workflow execution

Governance ensures agents operate safely and predictably.

Responsible AI Principles

Responsible AI principles apply strongly to AI agents.

Key principles include:

Fairness
Reliability
Privacy
Transparency
Accountability
Safety

Human Oversight

Human oversight is one of the most important governance mechanisms.

Humans may:

Approve actions
Review outputs
Escalate decisions
Override agent behavior

Oversight Modes

AI systems may use different oversight levels.

Common oversight modes include:

Human-in-the-loop
Human-on-the-loop
Human-out-of-the-loop

Human-in-the-Loop (HITL)

In HITL systems:

Humans approve important actions
Agents cannot complete tasks autonomously
Human validation is required

Examples:

Financial approvals
Healthcare decisions
Legal workflows

Human-on-the-Loop

In this model:

Agents operate autonomously
Humans monitor activity
Humans can intervene if needed

Examples:

Customer support routing
Workflow automation
Monitoring systems

Human-out-of-the-Loop

In this model:

Agents operate fully autonomously
No human review occurs during execution

This model introduces the highest risk.

Choosing Oversight Levels

Oversight requirements depend on:

Risk level
Regulatory requirements
Sensitivity of actions
Business impact

Higher-risk systems generally require stronger oversight.

Agent Constraints

Constraints limit what agents can do.

Constraints help:

Reduce harmful behavior
Prevent misuse
Enforce policy compliance

Types of Agent Constraints

Common constraints include:

Permission constraints
Data access restrictions
Tool restrictions
Workflow boundaries
Output limitations
Spending limits

Permission Constraints

Permission constraints limit:

Which systems agents can access
Which actions agents can perform

Example:

An agent may read customer data but cannot delete records.

Workflow Constraints

Workflow constraints restrict:

Multi-step actions
Automated decisions
Escalation capabilities

Example:

An agent may draft emails but require approval before sending them.

Tool-Access Controls

Tool-access controls regulate which tools agents can use.

This is a major AI-103 exam topic.

Why Tool Controls Matter

AI agents may access:

Databases
APIs
Email systems
Enterprise applications
External services

Without controls, agents could:

Expose sensitive data
Perform unauthorized actions
Cause operational damage

Least Privilege Access

Agents should receive only the minimum permissions required.

This follows the principle of least privilege.

Tool Allow Lists

Allow lists specify approved tools agents may access.

Benefits include:

Reduced attack surface
Improved governance
Better compliance

Tool Deny Lists

Deny lists block:

Dangerous tools
Unapproved APIs
Restricted workflows

Scoped Tool Permissions

Permissions may vary by:

User role
Workflow type
Business context
Risk level

Dynamic Tool Access

Some systems dynamically adjust permissions based on:

Risk assessments
User identity
Workflow conditions

Approval Workflows

Approval workflows require human validation before:

Tool execution
Sensitive actions
High-risk decisions

Examples of Approval Requirements

Examples include:

Financial transactions
HR changes
Legal communications
Customer account modifications

Safe Tool Execution

Safe execution mechanisms include:

Sandboxing
Rate limiting
Input validation
Output filtering
Action confirmation

Sandboxing

Sandboxing isolates agent operations from production systems.

Benefits include:

Reduced operational risk
Safer experimentation
Controlled testing

Prompt Injection Risks

Prompt injection attacks attempt to manipulate agent behavior.

Examples include:

Overriding instructions
Exposing secrets
Triggering unauthorized actions

Defending Against Prompt Injection

Defensive strategies include:

Instruction isolation
Input filtering
Content moderation
Tool restrictions
Approval workflows

Content Filtering

Content filtering helps prevent:

Harmful outputs
Toxic responses
Unsafe instructions

Azure AI Content Safety supports these capabilities.

Logging and Monitoring

Governed AI systems should log:

Tool usage
Agent decisions
Approval actions
Security events
Workflow execution

Audit Trails

Audit trails support:

Compliance
Security investigations
Governance reviews
Accountability

Transparency and Explainability

Organizations should understand:

Why agents made decisions
Which tools were used
Which data sources influenced outputs

Multi-Agent Systems

Multi-agent systems introduce additional governance complexity.

Challenges include:

Agent coordination
Cascading failures
Permission inheritance
Autonomous interactions

Governance for Multi-Agent Systems

Best practices include:

Clear role separation
Permission boundaries
Workflow isolation
Centralized monitoring

Risk-Based Governance

Governance strength should align with risk.

Low-risk tasks may allow:

Greater autonomy

High-risk tasks may require:

Human approval
Strict controls
Detailed auditing

Compliance and Governance Policies

Organizations may enforce policies for:

Data privacy
Regulatory compliance
Security standards
Ethical AI usage

Azure Governance Tools

Common Azure governance tools include:

Azure Policy
Azure Monitor
Microsoft Defender for Cloud
Azure API Management
Azure Key Vault

Securing Agent Memory and Knowledge

Agents may store:

Conversation history
User context
Retrieved knowledge

Organizations must secure:

Stored memory
Sensitive prompts
Retrieval pipelines

Data Minimization

Agents should access only the data required to complete tasks.

Benefits include:

Reduced risk
Improved privacy
Better compliance

Escalation Mechanisms

Agents should escalate:

High-risk requests
Ambiguous situations
Policy conflicts
Unsafe instructions

Fail-Safe Design

Fail-safe systems default to safe behavior when:

Errors occur
Permissions fail
Uncertainty is high

Common AI-103 Governance Scenarios

Scenario 1: Enterprise Financial Agent

Requirements:

Strict approvals
Transaction controls
Audit logging

Recommended Governance:

HITL workflows
Tool restrictions
Approval gates

Scenario 2: Customer Support Agent

Requirements:

Autonomous workflows
Limited customer data access
Escalation handling

Recommended Governance:

Scoped permissions
Human-on-the-loop oversight
Monitoring

Scenario 3: Internal Research Assistant

Requirements:

Knowledge retrieval
Read-only access
Grounded responses

Recommended Governance:

Retrieval restrictions
Private networking
Least privilege access

Scenario 4: Multi-Agent Workflow System

Requirements:

Coordinated automation
Controlled orchestration
Strong monitoring

Recommended Governance:

Permission boundaries
Centralized logging
Workflow isolation

Common AI-103 Exam Tips

Understand Oversight Models

Know the differences between:

Human-in-the-loop
Human-on-the-loop
Human-out-of-the-loop

Learn Tool Governance Concepts

Understand:

Tool restrictions
Allow lists
Scoped permissions
Approval workflows

Understand Responsible AI Principles

Know:

Transparency
Accountability
Safety
Privacy

Learn Security and Governance Best Practices

Understand:

Least privilege access
Logging and auditing
Prompt injection defenses
Risk-based governance

Summary

Governance is essential for safe and responsible AI agent systems.

For the AI-103 exam, you should understand:

Agent oversight modes
Human-in-the-loop workflows
Tool-access controls
Permission boundaries
Approval workflows
Prompt injection prevention
Logging and auditing
Responsible AI principles
Governance policies
Risk-based controls

Strong governance practices help ensure AI agents remain:

Safe
Reliable
Accountable
Compliant
Secure

These concepts are foundational for responsible AI deployment on Azure.

Practice Exam Questions

Question 1

Which oversight model requires human approval before an agent completes actions?

A. Human-out-of-the-loop
B. Human-on-the-loop
C. Human-in-the-loop
D. Fully autonomous mode

Answer

C. Human-in-the-loop

Explanation

Human-in-the-loop systems require human approval before execution.

Question 2

What is the primary purpose of tool-access controls?

A. Increase GPU utilization
B. Regulate which tools agents can use
C. Reduce storage redundancy
D. Improve network bandwidth

Answer

B. Regulate which tools agents can use

Explanation

Tool-access controls restrict tool usage and reduce risk.

Question 3

Which security principle grants agents only the permissions they require?

A. High availability
B. Least privilege
C. Semantic ranking
D. Horizontal scaling

Answer

B. Least privilege

Explanation

Least privilege minimizes unnecessary access.

Question 4

Which attack attempts to manipulate agent instructions?

A. Replication attack
B. Prompt injection attack
C. Scaling attack
D. Storage attack

Answer

B. Prompt injection attack

Explanation

Prompt injection attacks attempt to override system instructions.

Question 5

Which governance mechanism requires human approval before sensitive actions occur?

A. Vector indexing
B. Approval workflow
C. Semantic search
D. Batch processing

Answer

B. Approval workflow

Explanation

Approval workflows add human validation to high-risk actions.

Question 6

What is the purpose of sandboxing?

A. Increase token usage
B. Isolate agent operations from production systems
C. Reduce search relevance
D. Improve compression ratios

Answer

B. Isolate agent operations from production systems

Explanation

Sandboxing reduces operational risk during execution.

Question 7

Which oversight model allows autonomous operation while humans monitor activity?

A. Human-in-the-loop
B. Human-on-the-loop
C. Human-out-of-the-loop
D. Offline mode

Answer

B. Human-on-the-loop

Explanation

Humans supervise and may intervene when needed.

Question 8

What is a major benefit of audit trails?

A. Increased storage redundancy
B. Improved compliance and accountability
C. Reduced semantic ranking
D. Faster GPU performance

Answer

B. Improved compliance and accountability

Explanation

Audit trails support governance, investigations, and compliance.

Question 9

Which Azure service helps enforce governance policies?

A. Azure Policy
B. Azure CDN
C. Azure Files
D. Azure DNS

Answer

A. Azure Policy

Explanation

Azure Policy enforces governance and compliance standards.

Question 10

Why are allow lists useful for agent governance?

A. They increase network traffic
B. They restrict agents to approved tools
C. They reduce encryption
D. They eliminate monitoring requirements

Answer

B. They restrict agents to approved tools

Explanation

Allow lists reduce attack surface and improve governance.

Go to the AI-103 Exam Prep Hub main page

AI, AI Security, AI-103, Azure AI, Microsoft Certification May 25, 2026

Configure security, including managed identity, private networking, keyless credentials, and role policies (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Manage, monitor, and secure AI systems
      --> Configure security, including managed identity, private networking, keyless credentials, and role policies

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Security is one of the most important aspects of enterprise AI solutions.

AI applications often process:

Sensitive enterprise data
Proprietary documents
Customer information
Internal business knowledge
Regulated data

Modern AI systems may also:

Access external services
Execute tools
Use vector databases
Retrieve enterprise documents
Orchestrate AI agents

Because of this, organizations must secure:

AI models
APIs
Search services
Data sources
Agent workflows
Networking
Credentials
Access policies

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of AI security and governance on Azure.

For the AI-103 exam, you should understand:

Managed identities
Keyless authentication
Private networking
Role-Based Access Control (RBAC)
Role policies
Secure service access
Azure networking concepts
Authentication and authorization
Azure Key Vault
Network isolation
Secure AI architectures
Governance and compliance

Why AI Security Matters

AI systems introduce unique security risks.

Examples include:

Data leakage
Prompt injection attacks
Unauthorized tool execution
Credential exposure
Sensitive document access
API abuse
Model misuse

Security controls help:

Protect enterprise data
Enforce least privilege access
Reduce attack surfaces
Improve compliance
Secure AI workflows

Core Azure Security Concepts

Important Azure security concepts include:

Authentication
Authorization
Identity management
Network security
Secrets management
Access control
Governance

Authentication vs Authorization

Authentication verifies identity.

Examples:

User login
Service identity verification

Authorization determines permissions.

Examples:

Which resources users can access
What actions services can perform

Azure Entra ID

Azure Entra ID provides:

Identity management
Authentication
Access control
Enterprise security integration

Azure Entra ID is heavily used in Azure AI solutions.

Managed Identities

Managed identities provide secure identity management for Azure resources.

Managed identities eliminate the need to store credentials in code.

This is an extremely important AI-103 exam topic.

Why Managed Identities Matter

Without managed identities, developers may store:

API keys
Passwords
Secrets
Connection strings

This increases security risks.

Managed identities reduce these risks.

Types of Managed Identities

There are two main types:

System-assigned managed identities
User-assigned managed identities

System-Assigned Managed Identities

A system-assigned identity:

Is tied to one Azure resource
Is automatically managed by Azure
Is deleted when the resource is deleted

User-Assigned Managed Identities

A user-assigned identity:

Exists independently of resources
Can be shared across multiple services
Supports centralized identity management

Common Managed Identity Scenarios

Managed identities are commonly used when:

AI apps access Azure AI Search
AI agents access Blob Storage
Applications access Azure Key Vault
Services call Azure OpenAI

Keyless Credentials

Keyless authentication avoids hardcoded secrets.

Instead of API keys, systems use:

Managed identities
OAuth tokens
Azure Entra authentication

Benefits of Keyless Authentication

Benefits include:

Improved security
Reduced secret management
Automatic credential rotation
Lower risk of credential leaks

Azure Key Vault

Azure Key Vault securely stores:

Secrets
Keys
Certificates
Tokens

Using Key Vault with AI Solutions

AI applications commonly store:

API keys
Database credentials
Connection strings
Encryption keys

inside Key Vault.

Role-Based Access Control (RBAC)

RBAC controls who can access Azure resources.

RBAC uses:

Roles
Permissions
Scope assignments

Principle of Least Privilege

Least privilege means users and services receive only the permissions they need.

This reduces:

Security risks
Accidental misuse
Attack exposure

Common Azure Roles

Common built-in roles include:

Owner
Contributor
Reader
Cognitive Services User
Search Service Contributor

Custom Roles

Organizations may create custom roles with:

Specific permissions
Restricted access scopes

Scope Levels in RBAC

RBAC may apply at:

Management group level
Subscription level
Resource group level
Resource level

AI Role Policy Examples

Examples include:

Developers can deploy models
Analysts can query AI systems
Applications can access search indexes
Agents can retrieve documents

Network Security for AI Systems

AI systems often require secure networking.

Network security helps:

Prevent unauthorized access
Isolate resources
Protect sensitive data

Private Networking

Private networking isolates resources from the public internet.

This is heavily emphasized on AI-103.

Virtual Networks (VNets)

Azure Virtual Networks provide:

Network isolation
Secure communication
Controlled connectivity

Private Endpoints

Private endpoints allow services to be accessed privately through a VNet.

Benefits include:

Reduced internet exposure
Improved security
Private connectivity

Public vs Private Access

Public access:

Uses public internet endpoints
Easier to configure
Higher exposure risk

Private access:

Uses private network paths
Improves security
Supports enterprise compliance

Network Security Groups (NSGs)

NSGs control inbound and outbound traffic.

They support:

Traffic filtering
Security rules
Access restrictions

Firewalls

Azure Firewall helps secure:

Network traffic
Application traffic
Outbound internet access

Secure AI Architecture Example

An enterprise AI system may include:

Azure OpenAI Service
Azure AI Search
Blob Storage
Azure Key Vault
AI agents
VNets
Private endpoints

All connected through private networking.

Secure Agent-Based Systems

AI agents require additional security considerations.

Agents may:

Execute tools
Access APIs
Retrieve documents
Interact with databases

Agent Security Risks

Risks include:

Unauthorized actions
Excessive permissions
Data leakage
Prompt injection attacks

Securing Agent Workflows

Best practices include:

Least privilege access
Tool restrictions
Approval workflows
Logging and monitoring
Input validation

API Security

AI systems often expose APIs.

API security may include:

Authentication
Authorization
Rate limiting
API gateways
Monitoring

Azure API Management

Azure API Management helps:

Secure APIs
Enforce policies
Monitor usage
Apply throttling

Data Encryption

Encryption protects data:

At rest
In transit

Azure services support encryption by default.

TLS and HTTPS

TLS/HTTPS secure data transmitted across networks.

Secure AI systems should always use encrypted communication.

Compliance and Governance

Organizations may require compliance for:

Healthcare
Finance
Government
Enterprise security policies

Governance Policies

Governance may enforce:

Approved regions
Resource tagging
Security requirements
Allowed configurations

Azure Policy

Azure Policy helps enforce governance standards.

Examples include:

Requiring private endpoints
Blocking public access
Enforcing encryption

Monitoring Security Events

Organizations should monitor:

Failed authentication attempts
Unauthorized access
Suspicious activity
API abuse

Logging and Auditing

Logging supports:

Troubleshooting
Compliance
Security investigations
Audit trails

Security Monitoring Tools

Common tools include:

Azure Monitor
Microsoft Defender for Cloud
Application Insights
Azure Policy

Common AI-103 Security Scenarios

Scenario 1: Enterprise AI Chatbot

Requirements:

Secure document retrieval
Private networking
Keyless authentication

Recommended Security:

Managed identities
Private endpoints
RBAC

Scenario 2: Multi-Agent Enterprise Workflow

Requirements:

Controlled tool execution
Least privilege access
Workflow auditing

Recommended Security:

Custom roles
Logging
Approval controls

Scenario 3: Regulated Industry AI System

Requirements:

Compliance
Encryption
Restricted internet access

Recommended Security:

VNets
Private endpoints
Azure Policy

Scenario 4: Public AI API Platform

Requirements:

API protection
Usage monitoring
Abuse prevention

Recommended Security:

API Management
Rate limiting
Monitoring

Common AI-103 Exam Tips

Understand Managed Identities

Know:

System-assigned identities
User-assigned identities
Keyless authentication

Learn RBAC Concepts

Understand:

Roles
Permissions
Scope
Least privilege

Understand Private Networking

Know:

VNets
Private endpoints
Public vs private access

Learn Secure AI Architecture Principles

Understand:

Secret management
Encryption
Governance
Monitoring

Summary

Security is essential for enterprise AI and agent-based systems.

For the AI-103 exam, you should understand:

Managed identities
Keyless authentication
Azure Key Vault
RBAC and role policies
Private networking
VNets and private endpoints
API security
Secure AI architecture
Governance and compliance
Monitoring and auditing

Strong security practices help ensure AI systems remain:

Secure
Compliant
Reliable
Governed
Protected from misuse

These concepts are foundational for deploying secure AI solutions on Azure.

Practice Exam Questions

Question 1

What is a primary benefit of managed identities?

A. Increased GPU performance
B. Elimination of hardcoded credentials
C. Reduced network latency
D. Faster vector indexing

Answer

B. Elimination of hardcoded credentials

Explanation

Managed identities securely authenticate services without storing secrets in code.

Question 2

Which Azure service securely stores secrets and certificates?

A. Azure CDN
B. Azure Key Vault
C. Azure Files
D. Azure DNS

Answer

B. Azure Key Vault

Explanation

Azure Key Vault securely stores secrets, keys, and certificates.

Question 3

What is the difference between authentication and authorization?

A. Authentication manages networks, authorization manages storage
B. Authentication verifies identity, authorization controls permissions
C. Authentication encrypts data, authorization compresses data
D. Authentication handles backups, authorization handles monitoring

Answer

B. Authentication verifies identity, authorization controls permissions

Explanation

Authentication confirms identity, while authorization determines allowed actions.

Question 4

Which Azure networking feature enables private access to Azure services?

A. Public IP addresses
B. Private endpoints
C. DNS forwarding
D. Content delivery networks

Answer

B. Private endpoints

Explanation

Private endpoints allow secure private network connectivity.

Question 5

Which security principle grants only the permissions required to perform a task?

A. High availability
B. Least privilege
C. Horizontal scaling
D. Semantic ranking

Answer

B. Least privilege

Explanation

Least privilege minimizes security exposure.

Question 6

Which Azure service provides identity and access management?

A. Azure Entra ID
B. Azure CDN
C. Azure Monitor
D. Azure Backup

Answer

A. Azure Entra ID

Explanation

Azure Entra ID manages authentication and identity services.

Question 7

What is a major benefit of keyless authentication?

A. Increased storage costs
B. Reduced credential management risks
C. Lower vector search accuracy
D. Reduced encryption strength

Answer

B. Reduced credential management risks

Explanation

Keyless authentication reduces exposure to leaked secrets.

Question 8

Which Azure feature helps enforce governance requirements such as mandatory private endpoints?

A. Azure Policy
B. Azure CDN
C. Azure Files
D. Azure DNS

Answer

A. Azure Policy

Explanation

Azure Policy enforces governance and compliance standards.

Question 9

Which networking component filters inbound and outbound traffic?

A. Blob containers
B. Network Security Groups (NSGs)
C. Search indexes
D. Embedding models

Answer

B. Network Security Groups (NSGs)

Explanation

NSGs control network traffic through configurable rules.

Question 10

Which Azure service helps secure and manage APIs?

A. Azure API Management
B. Azure Files
C. Azure DNS
D. Azure Backup

Answer

A. Azure API Management

Explanation

Azure API Management secures APIs and applies usage policies.

Go to the AI-103 Exam Prep Hub main page

AI, AI-103, Azure AI, Microsoft Certification May 25, 2026

Monitor data ingestion quality, search index health, and relevance performance (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Manage, monitor, and secure AI systems
      --> Monitor data ingestion quality, search index health, and relevance performance

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI applications increasingly rely on Retrieval-Augmented Generation (RAG) systems and enterprise search solutions.

These systems commonly use:

Azure AI Search
Embedding models
Vector databases
Search indexes
Retrieval pipelines
Knowledge bases
Data ingestion workflows

The quality of AI responses depends heavily on:

Data ingestion quality
Search index health
Retrieval effectiveness
Relevance performance
Grounding quality

Even powerful Large Language Models (LLMs) can produce poor results if retrieval systems are inaccurate or unhealthy.

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of monitoring and maintaining retrieval and search systems.

For the AI-103 exam, you should understand:

Data ingestion pipelines
Search indexing
Azure AI Search monitoring
Vector indexing
Retrieval quality
Relevance evaluation
Search index optimization
Search performance monitoring
Grounding quality
Operational monitoring
Troubleshooting retrieval systems

Why Retrieval Monitoring Matters

AI systems often rely on external knowledge sources.

If retrieval systems fail:

Responses may become inaccurate
Hallucinations may increase
Grounding quality may decline
Users may lose trust

Monitoring retrieval systems helps ensure:

Reliable search results
Accurate grounding
Healthy indexes
High-quality responses

What Is Data Ingestion?

Data ingestion is the process of collecting and importing data into search and AI systems.

Common ingestion sources include:

PDFs
Websites
Databases
APIs
SharePoint
Blob Storage
Enterprise documents

Data Ingestion Pipelines

A typical ingestion pipeline includes:

Data extraction
Content transformation
Chunking
Embedding generation
Indexing
Metadata enrichment

Data Quality in AI Systems

Poor-quality data leads to:

Weak retrieval
Hallucinations
Irrelevant responses
Poor search rankings

Common Data Quality Issues

Examples include:

Missing data
Duplicate records
Corrupted files
Inconsistent formatting
Outdated documents
Incorrect metadata

Metadata Importance

Metadata improves retrieval and filtering.

Examples include:

Document titles
Authors
Categories
Dates
Security labels

Monitoring Data Ingestion Quality

Organizations should monitor:

Ingestion failures
Parsing errors
Duplicate content
Missing metadata
File processing errors
Embedding generation failures

Azure AI Search

Azure AI Search is a cloud-based search and retrieval platform.

It supports:

Full-text search
Vector search
Semantic search
Hybrid search
AI enrichment

Azure AI Search is heavily emphasized on AI-103.

Search Indexes

A search index stores searchable content.

Indexes may contain:

Text
Metadata
Embeddings
Vectors
Enriched content

What Is Index Health?

Index health refers to how well a search index functions.

Healthy indexes support:

Accurate retrieval
Fast search performance
High relevance
Reliable grounding

Common Index Health Issues

Examples include:

Stale indexes
Missing documents
Failed indexing jobs
Corrupted embeddings
Slow query performance
Fragmented indexes

Index Freshness

Freshness measures how current indexed data is.

Outdated indexes may produce:

Incorrect answers
Missing information
Reduced trust

Monitoring Index Updates

Organizations should monitor:

Indexing frequency
Indexing completion
Failed updates
Document synchronization

Incremental Indexing

Incremental indexing updates only changed content.

Benefits include:

Faster indexing
Reduced costs
Improved efficiency

Full Reindexing

Full reindexing rebuilds the entire index.

Used when:

Schema changes occur
Large data updates occur
Embedding models change

Schema Design

Index schemas define:

Searchable fields
Filterable fields
Sortable fields
Vector fields

Poor schema design can reduce:

Retrieval quality
Query performance
Relevance accuracy

Vector Search

Vector search uses embeddings to find semantically similar content.

Vector search is critical for:

RAG systems
Semantic retrieval
AI grounding

Embedding Quality

Embedding quality directly affects retrieval relevance.

Poor embeddings may cause:

Weak search matches
Irrelevant retrieval
Hallucinations

Monitoring Vector Indexes

Organizations should monitor:

Embedding generation success
Vector indexing completion
Query latency
Retrieval relevance

Semantic Search

Semantic search improves understanding of user intent.

Benefits include:

Better relevance
Improved ranking
More accurate retrieval

Hybrid Search

Hybrid search combines:

Keyword search
Vector search
Semantic ranking

Benefits include:

Improved accuracy
Better recall
More reliable grounding

Search Relevance Performance

Relevance measures how useful search results are.

High relevance improves:

User satisfaction
Grounding quality
AI response quality

Common Relevance Metrics

Important metrics include:

Precision
Recall
Mean Reciprocal Rank (MRR)
Relevance scores
Click-through rates

Precision

Precision measures how many retrieved results are relevant.

High precision means:

Fewer irrelevant results
Better grounding

Recall

Recall measures how many relevant documents are retrieved.

High recall reduces:

Missing information
Incomplete answers

Mean Reciprocal Rank (MRR)

MRR measures ranking quality.

Higher MRR means:

Relevant documents appear earlier in results

Grounding Quality and Search Relevance

Poor search relevance can cause:

Hallucinations
Unsupported claims
Incorrect answers

Strong retrieval improves grounding quality.

Chunking Strategies

Chunking divides documents into smaller pieces.

Chunk size affects:

Retrieval accuracy
Search relevance
Token usage
Grounding quality

Poor Chunking Problems

Poor chunking may:

Break context
Reduce relevance
Increase hallucinations

AI Enrichment Pipelines

Azure AI Search supports AI enrichment.

Enrichment may include:

OCR
Entity extraction
Key phrase extraction
Image analysis

Monitoring AI Enrichment

Organizations should monitor:

OCR failures
Enrichment latency
Extraction quality
Pipeline failures

Monitoring Search Performance

Search systems should be monitored for:

Latency
Throughput
Query failures
Slow responses
Resource consumption

Query Latency

Query latency measures search response time.

High latency may result from:

Large indexes
Poor query design
Heavy traffic
Complex vector searches

Capacity Planning

Search systems require sufficient capacity.

Considerations include:

Index size
Query volume
Concurrent users
Vector workloads

Scaling Azure AI Search

Scaling options include:

Additional replicas
Additional partitions

Replicas

Replicas improve:

Query throughput
Availability
Read performance

Partitions

Partitions improve:

Storage capacity
Index scalability
Large dataset handling

Monitoring and Observability Tools

Operational monitoring is essential.

Azure Monitor

Azure Monitor provides:

Metrics
Logs
Alerts
Diagnostics

Application Insights

Application Insights supports:

Request tracing
Performance monitoring
Error diagnostics

Logging Search Queries

Query logs help analyze:

Search behavior
Failed searches
Popular queries
Relevance problems

Dashboards and Alerts

Dashboards help visualize:

Query latency
Index health
Error rates
Retrieval quality

Alerts may notify teams when:

Indexing fails
Relevance declines
Latency spikes
Errors increase

Security and Compliance

Search systems may contain sensitive enterprise data.

Organizations should monitor:

Unauthorized access
Data leakage
Security policy violations

Access Control

Azure AI Search supports:

Role-Based Access Control (RBAC)
Authentication
Authorization

Common AI-103 Retrieval Scenarios

Scenario 1: Enterprise Knowledge Assistant

Requirements:

Strong grounding
High retrieval relevance
Current data

Recommended Monitoring:

Relevance metrics
Index freshness
Hallucination monitoring

Scenario 2: Large Document Repository

Requirements:

Large-scale indexing
Fast query performance
High availability

Recommended Monitoring:

Replicas and partitions
Query latency
Index growth

Scenario 3: Multimodal Search System

Requirements:

OCR quality
Embedding reliability
Search relevance

Recommended Monitoring:

Enrichment pipelines
Embedding generation
Vector search quality

Scenario 4: Public AI Search Portal

Requirements:

High concurrency
Cost management
Abuse protection

Recommended Monitoring:

API monitoring
Rate limiting
Query analytics

Common AI-103 Exam Tips

Understand Retrieval Fundamentals

Know:

Vector search
Semantic search
Hybrid search
RAG pipelines

Learn Relevance Metrics

Understand:

Precision
Recall
MRR
Ranking quality

Understand Search Scaling

Know the differences between:

Replicas
Partitions

Learn Monitoring Concepts

Understand:

Index health
Query latency
Retrieval quality
Data ingestion quality

Summary

Monitoring data ingestion quality, search index health, and relevance performance is critical for enterprise AI systems.

For the AI-103 exam, you should understand:

Data ingestion pipelines
Search indexing
Azure AI Search
Vector search
Retrieval monitoring
Relevance evaluation
Grounding quality
Search scaling
Monitoring tools
Operational best practices

Strong retrieval monitoring practices help ensure AI systems remain:

Accurate
Reliable
Grounded
Scalable
High performing

These concepts are foundational for Retrieval-Augmented Generation (RAG) and enterprise search systems on Azure.

Practice Exam Questions

Question 1

What is the primary purpose of a search index?

A. Encrypt network traffic
B. Store searchable content for retrieval
C. Compress application logs
D. Manage virtual machines

Answer

B. Store searchable content for retrieval

Explanation

Search indexes store searchable content, metadata, and vectors.

Question 2

Which Azure service is commonly used for vector search and semantic retrieval?

A. Azure AI Search
B. Azure DNS
C. Azure Backup
D. Azure Files

Answer

A. Azure AI Search

Explanation

Azure AI Search supports vector search, semantic search, and hybrid retrieval.

Question 3

What does index freshness measure?

A. Storage encryption
B. How current indexed data is
C. Network bandwidth
D. GPU utilization

Answer

B. How current indexed data is

Explanation

Fresh indexes contain the latest available information.

Question 4

Which metric measures how many retrieved documents are relevant?

A. Recall
B. Precision
C. Latency
D. Throughput

Answer

B. Precision

Explanation

Precision measures the percentage of relevant retrieved results.

Question 5

Which search approach combines vector search and keyword search?

A. Static search
B. Hybrid search
C. Batch search
D. Sequential search

Answer

B. Hybrid search

Explanation

Hybrid search combines semantic and keyword retrieval techniques.

Question 6

What is a common consequence of poor chunking?

A. Faster GPU performance
B. Reduced retrieval relevance
C. Increased network bandwidth
D. Lower storage capacity

Answer

B. Reduced retrieval relevance

Explanation

Poor chunking may break context and reduce retrieval quality.

Question 7

Which Azure AI Search scaling option improves query throughput and availability?

A. Partitions
B. Replicas
C. Firewalls
D. Load balancers

Answer

B. Replicas

Explanation

Replicas improve query performance and availability.

Question 8

Which metric measures how many relevant documents are successfully retrieved?

A. Precision
B. Recall
C. Latency
D. Error rate

Answer

B. Recall

Explanation

Recall measures how many relevant results are retrieved.

Question 9

Which Azure service provides metrics, logs, and alerts for operational monitoring?

A. Azure Monitor
B. Azure CDN
C. Azure DNS
D. Azure Backup

Answer

A. Azure Monitor

Explanation

Azure Monitor supports metrics, logging, and alerting.

Question 10

What is one major benefit of semantic search?

A. Increased hardware costs
B. Better understanding of user intent
C. Reduced storage redundancy
D. Lower network security

Answer

B. Better understanding of user intent

Explanation

Semantic search improves relevance by understanding query meaning.

Go to the AI-103 Exam Prep Hub main page

AI, AI-103, Azure AI, Microsoft Certification May 25, 2026

Monitor model performance, drift, safety events, and grounding quality (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Manage, monitor, and secure AI systems
      --> Monitor model performance, drift, safety events, and grounding quality

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI applications and agent-based systems require continuous monitoring and evaluation.

Unlike traditional applications, AI systems can change behavior over time due to:

Model drift
Data drift
Prompt changes
Retrieval issues
Tool failures
Safety risks
Hallucinations
Changes in user behavior

Organizations must monitor AI systems to ensure:

Reliability
Accuracy
Safety
Performance
Groundedness
Compliance
Cost efficiency

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of monitoring and operational management for AI systems.

For the AI-103 exam, you should understand:

AI observability concepts
Model performance monitoring
Drift detection
Safety monitoring
Grounding quality evaluation
Hallucination detection
Retrieval quality monitoring
Responsible AI practices
Logging and telemetry
Azure monitoring tools
Evaluation workflows

Why AI Monitoring Is Important

AI systems are probabilistic rather than deterministic.

This means:

Outputs can vary
Quality may fluctuate
Hallucinations may occur
Retrieval pipelines may fail
Safety risks may emerge

Continuous monitoring helps identify these issues early.

AI Observability

AI observability refers to understanding:

How AI systems behave
Why outputs are generated
Whether responses are accurate
Whether systems remain reliable over time

AI observability combines:

Metrics
Logging
Telemetry
Evaluation
Diagnostics

Model Performance Monitoring

Model performance monitoring measures how effectively AI systems perform tasks.

Common Performance Metrics

Common AI metrics include:

Accuracy
Precision
Recall
Latency
Throughput
Error rates
User satisfaction
Token usage

Latency Monitoring

Latency measures response time.

High latency may result from:

Large prompts
Large models
Slow retrieval
Tool execution delays
Heavy concurrency

Throughput Monitoring

Throughput measures how many requests a system can process.

Monitoring throughput helps:

Identify bottlenecks
Plan scaling
Optimize infrastructure

Error Rate Monitoring

Error monitoring tracks:

API failures
Timeout errors
Tool execution failures
Retrieval failures
Authentication errors

User Feedback Monitoring

User feedback helps evaluate:

Response quality
Relevance
Reliability
Satisfaction

Feedback may include:

Ratings
Surveys
Thumbs up/down systems

What Is Drift?

Drift occurs when system behavior changes over time.

Drift can reduce:

Accuracy
Reliability
Relevance

Types of Drift

Common types include:

Data drift
Concept drift
Model drift
Prompt drift

Data Drift

Data drift occurs when input data changes over time.

Examples:

New user behaviors
Different terminology
Seasonal patterns
Changing document formats

Concept Drift

Concept drift occurs when relationships between inputs and outputs change.

Example:

A fraud detection system may become less accurate as attack patterns evolve.

Model Drift

Model drift refers to declining model performance over time.

Causes may include:

Outdated training data
Changing business conditions
New vocabulary
Different workflows

Prompt Drift

Prompt drift occurs when prompt modifications unintentionally reduce quality.

Effects may include:

Increased hallucinations
Reduced consistency
Lower grounding quality

Drift Detection Techniques

Organizations may detect drift using:

Statistical analysis
Baseline comparisons
Evaluation datasets
Human review
Automated testing

Baseline Evaluation

Baseline evaluations establish reference performance metrics.

Future evaluations compare against the baseline.

Safety Monitoring

Safety monitoring is a major AI-103 exam topic.

AI systems must detect and mitigate:

Harmful content
Toxic responses
Bias
Jailbreak attempts
Prompt injection attacks
Unsafe outputs

Responsible AI Principles

Responsible AI principles include:

Fairness
Reliability
Privacy
Inclusiveness
Transparency
Accountability

Azure AI Content Safety

Azure AI Content Safety helps detect:

Hate speech
Violence
Self-harm content
Sexual content

Safety Events

Safety events include:

Harmful outputs
Unsafe prompts
Policy violations
Prompt injection attempts
Data leakage

Prompt Injection Attacks

Prompt injection attacks attempt to manipulate AI systems.

Examples include:

Ignoring instructions
Revealing confidential data
Executing unauthorized actions

Monitoring Prompt Injection

Detection strategies include:

Input filtering
Content moderation
Instruction isolation
Logging suspicious requests

Hallucinations

Hallucinations occur when models generate inaccurate or fabricated information.

Hallucinations are common risks in generative AI systems.

Causes of Hallucinations

Hallucinations may result from:

Weak retrieval
Missing grounding
Poor prompts
Insufficient context
Ambiguous requests

What Is Grounding?

Grounding connects AI responses to trusted data sources.

Grounding improves:

Accuracy
Reliability
Explainability
Trustworthiness

Retrieval-Augmented Generation (RAG)

RAG systems improve grounding by retrieving external knowledge before generating responses.

Common RAG components include:

Embedding models
Vector search
Azure AI Search
Knowledge bases

Grounding Quality Monitoring

Grounding quality measures whether responses are:

Supported by source data
Factually accurate
Relevant
Properly cited

Signs of Poor Grounding

Indicators include:

Unsupported claims
Fabricated citations
Irrelevant responses
Hallucinations
Incorrect facts

Retrieval Quality Monitoring

Retrieval quality directly affects grounding quality.

Poor retrieval may produce:

Irrelevant documents
Missing context
Incomplete answers

Important Retrieval Metrics

Common retrieval metrics include:

Recall
Precision
Relevance
Ranking quality

Chunking and Grounding

Chunking strategies affect retrieval quality.

Poor chunking may:

Break context
Reduce retrieval accuracy
Increase hallucinations

Human-in-the-Loop Evaluation

Human reviewers may evaluate:

Accuracy
Groundedness
Safety
Relevance
Bias

Human review is especially important for:

High-risk applications
Healthcare
Finance
Legal systems

Automated AI Evaluation

Automated evaluations help scale monitoring.

Evaluation systems may assess:

Toxicity
Groundedness
Relevance
Hallucination risk
Safety compliance

Prompt Flow Evaluation

Prompt Flow supports:

Workflow evaluation
Prompt testing
Automated scoring
AI experimentation

Prompt Flow is important for AI-103.

Logging and Telemetry

Logging helps organizations analyze system behavior.

Common logged information includes:

Requests
Responses
Errors
Latency
Token usage
Retrieval results

Azure Monitor

Azure Monitor provides:

Metrics
Logging
Alerts
Diagnostics

Application Insights

Application Insights supports:

Request tracing
Dependency monitoring
Performance analysis
Failure diagnostics

Alerting Systems

Alerts help teams respond quickly to issues.

Alerts may trigger when:

Error rates increase
Latency spikes
Safety violations occur
Costs exceed thresholds
Grounding quality declines

Dashboards and Visualization

Dashboards help teams visualize:

AI performance
System health
Usage patterns
Safety trends
Operational metrics

Monitoring Agent-Based Systems

AI agents introduce additional monitoring challenges.

Agents may involve:

Tool execution
Multi-step workflows
Retrieval pipelines
Autonomous decision-making

Agent Monitoring Metrics

Important metrics include:

Tool success rates
Workflow completion rates
Retrieval relevance
Conversation quality
Escalation frequency

Multi-Agent Systems

Multi-agent systems require monitoring for:

Coordination failures
Orchestration issues
Cascading errors
Excessive API usage

Compliance and Governance

Organizations may need compliance monitoring for:

Privacy regulations
Data retention
Responsible AI policies
Audit requirements

Security Monitoring

Security monitoring includes:

Authentication failures
Unauthorized access
Data leakage attempts
API abuse

Continuous Improvement

Monitoring supports continuous AI improvement.

Organizations may:

Refine prompts
Improve retrieval
Tune workflows
Retrain models
Adjust policies

Common AI-103 Monitoring Scenarios

Scenario 1: Enterprise Knowledge Assistant

Requirements:

Strong grounding
Reliable retrieval
Low hallucination rates

Recommended Monitoring:

Retrieval evaluation
Grounding metrics
Human review

Scenario 2: Public AI Chatbot

Requirements:

Safety monitoring
Abuse detection
Cost tracking

Recommended Monitoring:

Content Safety
API monitoring
Rate-limit alerts

Scenario 3: Multi-Agent Workflow Platform

Requirements:

Tool reliability
Workflow visibility
Performance monitoring

Recommended Monitoring:

Tool execution logs
Agent telemetry
Workflow dashboards

Scenario 4: Regulated Industry AI System

Requirements:

Compliance
Auditability
Human oversight

Recommended Monitoring:

Logging
Human review
Governance controls

Common AI-103 Exam Tips

Understand Drift Concepts

Know the differences between:

Data drift
Concept drift
Model drift
Prompt drift

Learn Grounding and Hallucination Concepts

Understand:

RAG
Retrieval quality
Hallucination causes
Grounded responses

Understand Responsible AI

Know:

Content Safety
Bias mitigation
Safety monitoring
Prompt injection risks

Know Monitoring Tools

Understand:

Azure Monitor
Application Insights
Prompt Flow
Azure AI Content Safety

Summary

Monitoring model performance, drift, safety events, and grounding quality is essential for enterprise AI systems.

For the AI-103 exam, you should understand:

AI observability
Performance metrics
Drift detection
Safety monitoring
Hallucination detection
Grounding quality
Retrieval evaluation
Logging and telemetry
Responsible AI practices
Monitoring tools and workflows

Strong monitoring practices help ensure AI systems remain:

Reliable
Accurate
Safe
Explainable
Compliant
High performing

These concepts are foundational for operational AI excellence on Azure.

Practice Exam Questions

Question 1

What is model drift?

A. Improved model accuracy over time
B. Declining model performance due to changing conditions
C. Increased network bandwidth
D. Reduced storage replication

Answer

B. Declining model performance due to changing conditions

Explanation

Model drift occurs when model behavior changes and performance degrades.

Question 2

Which Azure service helps detect harmful content in AI systems?

A. Azure AI Content Safety
B. Azure DNS
C. Azure Backup
D. Azure Files

Answer

A. Azure AI Content Safety

Explanation

Azure AI Content Safety detects harmful and unsafe content.

Question 3

What is grounding in generative AI?

A. Encrypting prompts
B. Connecting responses to trusted data sources
C. Increasing storage performance
D. Reducing network latency

Answer

B. Connecting responses to trusted data sources

Explanation

Grounding improves factual accuracy and reliability.

Question 4

Which issue occurs when an AI model generates fabricated information?

A. Autoscaling
B. Hallucination
C. Replication
D. Compression

Answer

B. Hallucination

Explanation

Hallucinations occur when AI systems generate false or unsupported information.

Question 5

Which type of drift occurs when input data changes over time?

A. Concept drift
B. Data drift
C. Prompt drift
D. Scaling drift

Answer

B. Data drift

Explanation

Data drift refers to changing input patterns or distributions.

Question 6

Which Azure service provides telemetry and diagnostics for AI applications?

A. Application Insights
B. Azure Firewall
C. Azure CDN
D. Azure Backup

Answer

A. Application Insights

Explanation

Application Insights supports monitoring and diagnostics.

Question 7

What is a common cause of hallucinations in RAG systems?

A. Strong retrieval quality
B. Missing or poor grounding
C. Low latency
D. Excessive monitoring

Answer

B. Missing or poor grounding

Explanation

Weak grounding increases hallucination risk.

Question 8

Which monitoring metric measures system response time?

A. Throughput
B. Recall
C. Latency
D. Precision

Answer

C. Latency

Explanation

Latency measures how quickly systems respond.

Question 9

Which attack attempts to manipulate AI system instructions?

A. SQL replication
B. Prompt injection attack
C. Vector indexing
D. Chunking attack

Answer

B. Prompt injection attack

Explanation

Prompt injection attempts to override system instructions.

Question 10

Which Azure tool supports AI workflow evaluation and prompt testing?

A. Prompt Flow
B. Azure CDN
C. Azure Firewall
D. Azure Backup

Answer

A. Prompt Flow

Explanation

Prompt Flow supports workflow orchestration and evaluation.

Go to the AI-103 Exam Prep Hub main page

AI, AI-103, Azure AI, Microsoft Certification May 25, 2026

Manage quotas, scaling, rate limits, and cost footprints for model and agent workloads (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Manage, monitor, and secure AI systems
      --> Manage quotas, scaling, rate limits, and cost footprints for model and agent workloads

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI applications and agent-based systems can consume significant compute resources and operational costs.

Generative AI workloads often involve:

Large Language Models (LLMs)
Embedding generation
Vector search
Retrieval-Augmented Generation (RAG)
AI agents
Tool execution
Workflow orchestration
Multimodal processing

As AI applications scale, organizations must carefully manage:

Quotas
Throughput limits
Rate limits
Token usage
Infrastructure scaling
Operational costs
Resource utilization

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of how to manage and optimize AI workloads in Azure.

For the AI-103 exam, you should understand:

Quota management
Rate limiting
Scaling strategies
Throughput optimization
Cost optimization
Monitoring AI workloads
Autoscaling
Capacity planning
Token management
Model selection tradeoffs
Agent workload optimization

Understanding AI Workload Consumption

AI workloads consume resources differently than traditional applications.

Key consumption factors include:

Prompt size
Response size
Number of requests
Model size
Embedding generation
Retrieval operations
Concurrent users
Tool execution

Tokens and Token Consumption

Generative AI models process text using tokens.

Tokens represent:

Words
Word fragments
Characters
Symbols

Token usage directly affects:

Cost
Latency
Throughput
Performance

Input Tokens

Input tokens include:

User prompts
System prompts
Retrieved documents
Conversation history

Output Tokens

Output tokens represent generated responses.

Longer responses increase:

Costs
Latency
Resource consumption

Context Windows

A context window is the amount of information a model can process in a request.

Larger context windows:

Support more information
Increase token consumption
Increase costs
Potentially increase latency

What Are Quotas?

Quotas define resource usage limits for Azure AI services.

Quotas help:

Prevent overconsumption
Ensure fair resource usage
Protect service reliability

Common Azure AI Quotas

Common quotas include:

Requests per minute (RPM)
Tokens per minute (TPM)
Concurrent requests
Deployment limits
Resource limits

Requests Per Minute (RPM)

RPM limits how many API requests can be processed each minute.

High request volumes may require:

Additional deployments
Provisioned throughput
Load balancing

Tokens Per Minute (TPM)

TPM limits the number of tokens processed per minute.

High-token workloads often require:

Throughput optimization
Smaller prompts
Efficient retrieval
Better chunking strategies

Provisioned Throughput

Provisioned throughput reserves dedicated model capacity.

Benefits include:

Predictable performance
Consistent latency
Higher throughput

Tradeoffs include:

Higher cost
Capacity planning requirements

Standard Deployments vs Provisioned Throughput

Standard Deployments

Advantages:

Lower cost
Flexible scaling
Simpler management

Disadvantages:

Shared capacity
Less predictable latency

Provisioned Throughput Deployments

Advantages:

Dedicated capacity
Predictable performance
Enterprise reliability

Disadvantages:

Higher cost
Requires workload planning

Rate Limiting

Rate limiting controls how frequently clients can access services.

Benefits include:

Preventing abuse
Improving stability
Protecting infrastructure

Why Rate Limits Matter

Without rate limits:

Services may become overloaded
Costs may increase rapidly
Applications may experience outages

Handling Rate Limit Errors

Applications should gracefully handle rate limit responses.

Common strategies include:

Retry policies
Exponential backoff
Queueing
Load balancing

Exponential Backoff

Exponential backoff increases wait times between retries.

Benefits:

Reduces service overload
Improves reliability
Helps recover from temporary spikes

Queue-Based Architectures

Queues help manage burst traffic.

Common Azure services include:

Azure Service Bus
Azure Queue Storage

Benefits:

Improved reliability
Controlled workload processing
Better scalability

Scaling AI Workloads

AI systems must scale efficiently.

Horizontal Scaling

Horizontal scaling adds more instances.

Examples:

Additional containers
More API instances
More worker nodes

Benefits:

Better concurrency
Higher throughput
Improved resilience

Vertical Scaling

Vertical scaling increases resource capacity.

Examples:

More CPU
More memory
Larger compute sizes

Autoscaling

Autoscaling dynamically adjusts resources based on workload demand.

Common Azure services supporting autoscaling:

AKS
Azure Functions
Azure App Service
Azure Container Apps

Scaling AI Agents

AI agents often require additional scaling considerations.

Agent workloads may involve:

Tool execution
Retrieval pipelines
Multi-step reasoning
Long-running workflows

Multi-Agent Systems

Multi-agent systems may generate:

High API volumes
Increased orchestration complexity
Heavy retrieval traffic

Scaling strategies may include:

Distributed architectures
Queue systems
Parallel processing

Cost Footprints for AI Systems

AI systems can become expensive very quickly.

Common AI Cost Drivers

Major cost drivers include:

Token usage
Large models
Embedding generation
Vector search
Provisioned throughput
Storage
Networking
Agent orchestration

Large Models vs Small Models

Large Models

Advantages:

Better reasoning
Higher-quality responses
Stronger generalization

Disadvantages:

Higher costs
Increased latency
Greater resource consumption

Small Models

Advantages:

Lower cost
Faster responses
Reduced latency

Disadvantages:

Reduced reasoning capability
Less sophisticated outputs

Choosing the Right Model

Choose smaller models when:

Tasks are simple
Low latency matters
Budget constraints exist

Choose larger models when:

Advanced reasoning is required
Complex workflows exist
Higher quality is critical

Optimizing Prompt Design

Prompt design directly affects cost.

Long prompts:

Increase token usage
Increase latency
Increase costs

Prompt Optimization Strategies

Strategies include:

Shorter prompts
Better instructions
Efficient context usage
Retrieval filtering
Context summarization

Retrieval Optimization

RAG systems can significantly increase token usage.

Retrieved documents consume context window space.

Chunking Optimization

Chunking strategies affect:

Retrieval accuracy
Token consumption
Latency

Poor chunking may:

Increase irrelevant retrieval
Increase costs
Reduce quality

Hybrid Search Optimization

Hybrid search combines:

Vector search
Keyword search

Benefits include:

Better retrieval accuracy
Reduced hallucinations
More relevant grounding

Monitoring AI Workloads

Monitoring is essential for operational management.

Azure Monitor

Azure Monitor provides:

Metrics
Alerts
Logs
Diagnostics

Application Insights

Application Insights supports:

Telemetry
Request tracing
Dependency monitoring
Performance analysis

Important Metrics to Monitor

Common AI metrics include:

Token usage
Latency
Error rates
Throughput
Cost trends
Retrieval quality
Tool execution failures

Cost Monitoring

Organizations should track:

Daily usage
Monthly spend
Per-user costs
Per-agent costs
API consumption

Azure Cost Management

Azure Cost Management helps:

Analyze spending
Forecast costs
Create budgets
Detect anomalies

Budget Alerts

Budget alerts notify teams when spending thresholds are exceeded.

Benefits include:

Better cost control
Early detection of anomalies
Prevention of runaway spending

Security and Cost Protection

Security issues can increase costs.

Examples include:

API abuse
Prompt injection attacks
Excessive automated requests

API Management

Azure API Management helps:

Apply throttling
Control rate limits
Secure APIs
Monitor usage

Caching Strategies

Caching reduces repeated AI calls.

Benefits include:

Reduced token usage
Lower latency
Lower costs

Common Caching Scenarios

Cache:

Frequent responses
Static retrieval results
Reusable embeddings
Common prompts

High Availability Considerations

Scaling should also support:

Reliability
Fault tolerance
Disaster recovery

Load Balancing

Load balancing distributes requests across instances.

Benefits:

Improved scalability
Better resilience
Higher throughput

Common AI-103 Operational Scenarios

Scenario 1: Enterprise AI Copilot

Requirements:

High concurrency
Predictable latency
Cost monitoring

Recommended Strategy:

Provisioned throughput
Autoscaling
Budget alerts

Scenario 2: Internal Knowledge Assistant

Requirements:

Retrieval optimization
Controlled costs
Moderate scale

Recommended Strategy:

Efficient chunking
Hybrid search
Smaller embedding models

Scenario 3: Multi-Agent Workflow Platform

Requirements:

Heavy orchestration
Parallel execution
High throughput

Recommended Strategy:

Queue-based architecture
AKS autoscaling
API throttling

Scenario 4: Public AI Chatbot

Requirements:

Abuse protection
Traffic spikes
Cost protection

Recommended Strategy:

API Management
Rate limiting
Caching
Autoscaling

Common AI-103 Exam Tips

Understand Quota Concepts

Know:

RPM limits
TPM limits
Provisioned throughput
Concurrent request limits

Understand Scaling Strategies

Know the differences between:

Horizontal scaling
Vertical scaling
Autoscaling

Learn Cost Optimization Techniques

Understand:

Prompt optimization
Model selection
Retrieval optimization
Caching
Budget monitoring

Know Monitoring and Operational Management

Understand:

Azure Monitor
Application Insights
Azure Cost Management
API Management

Summary

Managing quotas, scaling, rate limits, and cost footprints is essential for production AI systems.

For the AI-103 exam, you should understand:

Token consumption
Quota management
Throughput planning
Rate limiting
Scaling strategies
Cost optimization
Retrieval optimization
Monitoring AI workloads
Budget management
Operational resilience

Strong operational management practices help ensure AI systems remain:

Reliable
Scalable
Cost-effective
Secure
High performing

These concepts are critical for enterprise AI applications and agent-based solutions on Azure.

Practice Exam Questions

Question 1

What does TPM stand for in Azure AI workloads?

A. Tokens Per Minute
B. Tasks Per Model
C. Throughput Per Memory
D. Transactions Per Model

Answer

A. Tokens Per Minute

Explanation

TPM measures how many tokens can be processed each minute.

Question 2

Which deployment option provides dedicated processing capacity?

A. Shared deployment
B. Provisioned throughput deployment
C. Standard deployment
D. Public deployment

Answer

B. Provisioned throughput deployment

Explanation

Provisioned throughput reserves dedicated model capacity.

Question 3

What is the primary purpose of rate limiting?

A. Increase latency
B. Prevent abuse and protect services
C. Reduce storage replication
D. Encrypt prompts

Answer

B. Prevent abuse and protect services

Explanation

Rate limiting helps maintain service stability and prevent overload.

Question 4

Which retry strategy gradually increases wait times between retries?

A. Static retry
B. Exponential backoff
C. Parallel retry
D. Immediate retry

Answer

B. Exponential backoff

Explanation

Exponential backoff reduces overload during retry attempts.

Question 5

Which scaling strategy adds more instances to support increased workloads?

A. Vertical scaling
B. Horizontal scaling
C. Static scaling
D. Semantic scaling

Answer

B. Horizontal scaling

Explanation

Horizontal scaling increases capacity by adding instances.

Question 6

Which Azure service helps analyze and forecast cloud spending?

A. Azure Cost Management
B. Azure CDN
C. Azure Backup
D. Azure DNS

Answer

A. Azure Cost Management

Explanation

Azure Cost Management provides spending analysis and budgeting.

Question 7

What is one benefit of caching AI responses?

A. Increased token usage
B. Reduced costs and latency
C. Higher embedding size
D. Reduced monitoring

Answer

B. Reduced costs and latency

Explanation

Caching avoids repeated AI calls and improves performance.

Question 8

Which Azure service supports API throttling and traffic control?

A. Azure API Management
B. Azure Files
C. Azure DNS
D. Azure Backup

Answer

A. Azure API Management

Explanation

Azure API Management supports throttling, monitoring, and API governance.

Question 9

Which factor directly increases token consumption in generative AI systems?

A. Smaller prompts
B. Longer prompts and responses
C. Lower concurrency
D. Reduced context windows

Answer

B. Longer prompts and responses

Explanation

Larger prompts and outputs consume more tokens.

Question 10

Which Azure monitoring service provides telemetry and diagnostics for AI applications?

A. Application Insights
B. Azure Firewall
C. Azure CDN
D. Azure Files

Answer

A. Application Insights

Explanation

Application Insights provides telemetry, diagnostics, and performance monitoring.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Artificial Intelligence (AI), Azure AI, Generative AI, Microsoft Certification May 25, 2026

Design workflows, tool-augmented flows, and multistep reasoning pipelines (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build generative applications by using Foundry
      --> Design workflows, tool-augmented flows, and multistep reasoning pipelines

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI systems are evolving beyond simple prompt-response interactions.

Today’s generative AI applications often:

Use external tools
Perform multistep reasoning
Orchestrate workflows
Retrieve enterprise data
Execute actions autonomously
Coordinate across services

These systems are commonly called:

Agentic systems
Tool-augmented AI systems
AI workflow pipelines

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of designing intelligent workflows and reasoning pipelines.

For the AI-103 exam, you should understand:

AI workflows
Agent orchestration
Tool augmentation
Function calling
Multistep reasoning
Workflow pipelines
Retrieval integration
Memory integration
Planning and execution
Human-in-the-loop workflows
Monitoring and governance

What Are AI Workflows?

AI workflows are structured sequences of operations that combine:

AI reasoning
Data retrieval
Tool execution
Decision-making
Automation

Workflows coordinate multiple steps to complete complex tasks.

Why AI Workflows Matter

Simple prompts are often insufficient for:

Enterprise automation
Complex reasoning
Dynamic decision-making
Multi-system integration

Workflows allow AI systems to:

Break problems into steps
Use external tools
Validate outputs
Iterate toward solutions

What Is Tool Augmentation?

Tool augmentation allows AI systems to use external capabilities.

Examples include:

APIs
Databases
Search engines
Calculators
Business systems
Code interpreters

Why Tool Augmentation Is Important

Language models alone:

Cannot access real-time data
Cannot execute business actions directly
Cannot reliably perform all calculations

Tools extend AI capabilities.

Common Tool-Augmented Scenarios

Examples include:

Checking inventory
Booking appointments
Querying databases
Sending emails
Executing workflows
Calling REST APIs

What Is Function Calling?

Function calling enables models to:

Detect when a tool is needed
Generate structured tool requests
Invoke external services
Process returned results

Function Calling Workflow

Typical flow:

User submits request
Model determines tool requirement
Model generates function call
External tool executes
Results return to model
Model generates final response

Structured Tool Inputs

Function calling typically uses:

JSON schemas
Structured parameters
Validated inputs

This improves reliability.

Tool Selection

Agentic systems may dynamically choose:

Which tools to use
Which workflows to invoke
Which retrieval strategies to apply

Tool Orchestration

Tool orchestration coordinates multiple tools within a workflow.

Examples include:

Retrieval + summarization
Search + booking systems
Database queries + reporting

Sequential Workflows

Sequential workflows execute steps in order.

Example:

Retrieve customer data
Analyze account status
Generate recommendations
Send response

Parallel Workflows

Parallel workflows execute multiple tasks simultaneously.

Benefits include:

Faster execution
Better scalability
Reduced latency

Conditional Workflows

Conditional workflows branch based on:

User intent
Retrieved data
Safety evaluations
Confidence scores

What Is Multistep Reasoning?

Multistep reasoning breaks complex problems into smaller steps.

This improves:

Accuracy
Planning
Decision quality

Examples of Multistep Reasoning

Examples include:

Research workflows
Financial analysis
Travel planning
Technical troubleshooting

Chain-of-Thought Reasoning

Chain-of-thought reasoning encourages models to:

Reason step-by-step
Decompose problems
Validate intermediate steps

Planning and Execution Models

Agentic systems often separate:

Planning
Execution

The planner decides:

What steps are needed
Which tools to use

The executor performs actions.

Planner-Executor Architectures

Planner-executor architectures support:

Dynamic workflows
Adaptive reasoning
Task decomposition

ReAct Pattern

The ReAct (Reason + Act) pattern combines:

Reasoning
Tool usage
Observation
Iterative decision-making

Reflection and Self-Correction

Some systems support:

Self-evaluation
Output refinement
Error correction

Retrieval-Augmented Workflows

Workflows often integrate:

Vector search
RAG pipelines
Enterprise grounding

Memory in Agentic Systems

AI systems may use memory for:

Conversation history
User preferences
Workflow state
Long-running tasks

Short-Term Memory

Short-term memory stores:

Current conversation context
Immediate workflow information

Long-Term Memory

Long-term memory stores:

Persistent preferences
Historical interactions
Learned context

Workflow State Management

State management tracks:

Current task progress
Intermediate outputs
Pending actions

Human-in-the-Loop (HITL) Workflows

High-risk workflows may require:

Human approvals
Validation checkpoints
Escalation paths

Approval Gates

Approval gates can prevent:

Unsafe actions
Unauthorized tool usage
Harmful outputs

Safety and Governance

Organizations should enforce:

Tool restrictions
Permission boundaries
Safety filters
Approval workflows

Autonomous vs Semi-Autonomous Agents

Autonomous Agents

Can:

Make decisions independently
Execute workflows automatically

Semi-Autonomous Agents

Require:

Human review
Approval checkpoints

Workflow Monitoring

Organizations should monitor:

Tool usage
Failures
Safety violations
Latency
Costs

Trace Logging

Trace logging helps track:

Workflow execution
Tool calls
Reasoning steps
Agent decisions

Error Handling in Workflows

Workflow pipelines should handle:

API failures
Missing data
Timeout errors
Invalid outputs

Retry Strategies

Common retry strategies include:

Automatic retries
Fallback workflows
Alternative tool selection

Fallback Models

Applications may use fallback models when:

Primary models fail
Costs exceed thresholds
Latency becomes excessive

Workflow Optimization

Optimization strategies include:

Parallel processing
Caching
Smaller models
Efficient retrieval

Latency Considerations

Complex workflows may increase latency due to:

Multiple model calls
Tool invocations
Retrieval operations

Cost Considerations

Tool-augmented systems may increase:

Token usage
API calls
Infrastructure costs

Azure AI Foundry Workflow Capabilities

Azure AI Foundry supports:

Model orchestration
Tool integration
Agent workflows
Evaluation pipelines
Monitoring

Common AI-103 Workflow Scenarios

Scenario 1: Enterprise Research Assistant

Requirements:

Multi-document retrieval
Summarization
Citation generation

Recommended Workflow:

RAG + multistep reasoning

Scenario 2: Customer Service Agent

Requirements:

CRM access
Ticket management
Escalation workflows

Recommended Workflow:

Tool-augmented agent

Scenario 3: Financial Approval System

Requirements:

Risk evaluation
Human approvals
Audit logging

Recommended Workflow:

HITL approval pipeline

Scenario 4: AI Coding Assistant

Requirements:

Code generation
Code execution
Documentation retrieval

Recommended Workflow:

Code model + tool orchestration

Common AI-103 Exam Tips

Understand Workflow Patterns

Know:

Sequential workflows
Parallel workflows
Conditional workflows

Learn Tool-Augmented AI Concepts

Understand:

Function calling
Tool orchestration
Dynamic tool selection

Understand Multistep Reasoning

Know:

Chain-of-thought reasoning
Planner-executor patterns
ReAct workflows

Learn Governance Concepts

Understand:

HITL workflows
Approval gates
Monitoring
Trace logging

Summary

Modern AI applications increasingly rely on:

Workflow orchestration
Tool augmentation
Multistep reasoning
Agentic architectures

For the AI-103 exam, you should understand:

AI workflow design
Function calling
Tool orchestration
Sequential and parallel workflows
Multistep reasoning
Planner-executor architectures
ReAct patterns
Memory integration
HITL workflows
Monitoring and governance

These concepts enable organizations to build:

Intelligent
Autonomous
Scalable
Governed AI systems

They are foundational for modern generative AI and agentic solutions on Azure.

Practice Exam Questions

Question 1

What is the primary purpose of tool augmentation in AI systems?

A. Reduce storage costs
B. Extend model capabilities using external tools
C. Eliminate prompts
D. Replace vector search

Answer

B. Extend model capabilities using external tools

Explanation

Tool augmentation enables AI systems to interact with APIs, databases, and other services.

Question 2

What does function calling enable a model to do?

A. Generate only static responses
B. Invoke external tools using structured inputs
C. Eliminate workflows
D. Replace embeddings

Answer

B. Invoke external tools using structured inputs

Explanation

Function calling allows models to interact with external services.

Question 3

Which workflow type executes tasks simultaneously?

A. Sequential workflow
B. Parallel workflow
C. Manual workflow
D. Static workflow

Answer

B. Parallel workflow

Explanation

Parallel workflows improve speed by running tasks concurrently.

Question 4

What is multistep reasoning?

A. Compressing vector indexes
B. Breaking complex tasks into smaller reasoning steps
C. Increasing GPU memory
D. Reducing prompt size only

Answer

B. Breaking complex tasks into smaller reasoning steps

Explanation

Multistep reasoning improves problem-solving accuracy.

Question 5

What does the ReAct pattern combine?

A. Compression and storage
B. Reasoning and acting
C. Replication and scaling
D. Encryption and backup

Answer

B. Reasoning and acting

Explanation

ReAct combines reasoning steps with tool usage.

Question 6

What is the purpose of workflow state management?

A. Monitor GPU temperature
B. Track task progress and intermediate outputs
C. Disable logging
D. Replace semantic search

Answer

B. Track task progress and intermediate outputs

Explanation

State management helps maintain workflow continuity.

Question 7

Which architecture separates planning from execution?

A. Static inference architecture
B. Planner-executor architecture
C. Batch storage architecture
D. Compression architecture

Answer

B. Planner-executor architecture

Explanation

Planner-executor systems divide reasoning and execution responsibilities.

Question 8

Why are approval gates important in AI workflows?

A. They increase vector dimensions
B. They prevent unsafe or unauthorized actions
C. They reduce indexing speed
D. They eliminate monitoring requirements

Answer

B. They prevent unsafe or unauthorized actions

Explanation

Approval gates enforce governance and human oversight.

Question 9

Which concept allows AI systems to remember previous interactions?

A. Semantic ranking
B. Memory integration
C. Static chunking
D. GPU partitioning

Answer

B. Memory integration

Explanation

Memory enables contextual continuity and long-running workflows.

Question 10

What is a major challenge of complex AI workflows?

A. Eliminating all costs
B. Increased latency from multiple operations
C. Removing all need for monitoring
D. Preventing all hallucinations automatically

Answer

B. Increased latency from multiple operations

Explanation

Complex workflows may require multiple model calls and tool executions.

Go to the AI-103 Exam Prep Hub main page

AI, AI-103, Microsoft Certification May 25, 2026

Integrate Foundry projects with Continuous Integration and Continuous Deployment (CI/CD) pipelines (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Set up AI solutions in Foundry
      --> Integrate Foundry projects with Continuous Integration and Continuous Deployment (CI/CD) pipelines

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI applications and agent-based systems are continuously evolving.

Organizations frequently update:

AI models
Prompts
Agent workflows
APIs
Retrieval systems
Infrastructure
Security configurations

Manual deployment processes are slow, error-prone, and difficult to scale.

To solve these challenges, organizations use:

Continuous Integration (CI)
Continuous Deployment (CD)
Automated testing
Infrastructure-as-Code (IaC)
Automated validation pipelines

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of how to integrate Azure AI Foundry projects into CI/CD pipelines.

For the AI-103 exam, you should understand:

CI/CD concepts
Azure DevOps pipelines
GitHub Actions workflows
Infrastructure-as-Code
Automated AI deployment workflows
Model versioning
Deployment automation
Testing and validation
Environment management
Rollback strategies
Monitoring deployment health

What Is CI/CD?

CI/CD stands for:

Continuous Integration
Continuous Deployment (or Continuous Delivery)

CI/CD automates software and AI deployment processes.

Continuous Integration (CI)

Continuous Integration focuses on:

Automatically building code
Running automated tests
Validating changes
Detecting issues early

Developers frequently merge changes into shared repositories.

Continuous Deployment (CD)

Continuous Deployment automates:

Application releases
Model deployments
Infrastructure updates
Environment promotion

CD ensures new versions are deployed safely and consistently.

Why CI/CD Matters for AI Solutions

AI systems are more complex than traditional applications because they include:

Models
Prompts
Retrieval pipelines
Vector indexes
Agent workflows
Tool integrations

CI/CD helps ensure:

Reliable deployments
Repeatable processes
Faster releases
Reduced downtime
Safer experimentation

Azure AI Foundry and CI/CD

Azure AI Foundry integrates with:

Azure DevOps
GitHub Actions
Infrastructure-as-Code tools
Azure CLI
SDKs
REST APIs

This enables automated AI workflows.

Source Control for AI Projects

AI projects should use source control systems.

Common repositories include:

GitHub
Azure Repos

What Should Be Stored in Source Control?

Common AI assets include:

Application code
Prompt templates
Agent configurations
Infrastructure definitions
Deployment scripts
Evaluation workflows
Test cases
CI/CD pipeline definitions

What Should NOT Be Stored in Source Control?

Never store:

Secrets
API keys
Passwords
Certificates
Sensitive credentials

Use Azure Key Vault instead.

Azure DevOps

Azure DevOps provides:

Repositories
Build pipelines
Release pipelines
Work tracking
Artifact management

Azure DevOps is commonly used for enterprise AI deployments.

GitHub Actions

GitHub Actions supports:

Automated workflows
Build automation
Testing pipelines
Deployment automation
CI/CD orchestration

GitHub Actions is widely used for AI applications hosted in GitHub repositories.

Infrastructure-as-Code (IaC)

Infrastructure-as-Code automates infrastructure provisioning.

Instead of manually creating resources, infrastructure is defined in code.

Benefits of IaC

IaC provides:

Repeatability
Version control
Consistency
Automation
Reduced configuration drift

Common IaC Tools in Azure

Common Azure IaC tools include:

ARM templates
Bicep
Terraform

Bicep

Bicep is a declarative language for Azure infrastructure.

Used to deploy:

Azure OpenAI resources
Azure AI Search
Storage accounts
Networking resources
Key Vault
App Services

Terraform

Terraform is a multi-cloud Infrastructure-as-Code tool.

Useful for:

Hybrid environments
Multi-cloud deployments
Large enterprise automation

Automating Azure AI Resource Deployment

CI/CD pipelines can automatically provision:

Azure OpenAI
Azure AI Search
Cosmos DB
Azure Functions
App Service
Networking
Monitoring services

Automating Model Deployments

Model deployment pipelines may automate:

Model version selection
Deployment creation
Endpoint configuration
Scaling configuration
Rollback management

Model Versioning

Versioning is critical for AI deployments.

Benefits include:

Safer updates
Rollback support
Testing new versions
Comparing performance

Environment Management

AI solutions commonly use multiple environments.

Typical environments include:

Development
Testing
Staging
Production

Development Environment

Used for:

Experimentation
Initial testing
Prompt development
Rapid iteration

Testing Environment

Used for:

Automated testing
Integration testing
Validation workflows

Staging Environment

Used for:

Final validation
Production-like testing
User acceptance testing

Production Environment

Used for:

Live workloads
Enterprise applications
Customer-facing systems

Production environments require:

Strong monitoring
Security controls
Scalability
High availability

Automated Testing in AI Pipelines

Testing AI systems is more complex than traditional software testing.

AI pipelines should validate:

Functional behavior
Prompt quality
Retrieval quality
Latency
Safety
Reliability

Unit Testing

Unit testing validates:

Individual functions
APIs
Tool integrations
Components

Integration Testing

Integration testing validates interactions between:

Models
APIs
Search systems
Databases
Agents

Prompt Evaluation

Prompt evaluation helps assess:

Response quality
Groundedness
Hallucinations
Relevance
Consistency

Automated Evaluation Pipelines

Evaluation pipelines may measure:

Accuracy
Latency
Token usage
Toxicity
Retrieval precision

Prompt Flow and CI/CD

Prompt Flow can integrate into CI/CD pipelines.

Prompt Flow supports:

Workflow orchestration
Evaluation pipelines
Prompt testing
Tool integration

Deployment Strategies

Safe deployment strategies reduce risk.

Blue-Green Deployments

Blue-green deployments use two environments:

Current production environment
New deployment environment

Traffic switches after validation.

Benefits:

Reduced downtime
Easy rollback
Safer deployments

Canary Deployments

Canary deployments release updates gradually.

Benefits:

Reduced deployment risk
Easier issue detection
Controlled rollout

Rolling Deployments

Rolling deployments update systems incrementally.

Benefits:

Minimal downtime
Gradual infrastructure replacement

Rollback Strategies

Rollback mechanisms are critical.

Rollbacks may restore:

Previous model versions
Prior prompts
Earlier infrastructure states

Deployment Approval Gates

Approval gates help control production releases.

Approvals may be required before:

Production deployment
Model upgrades
Infrastructure changes

Security in CI/CD Pipelines

Security is a major AI-103 topic.

Azure Key Vault Integration

Pipelines should retrieve secrets securely from:

Azure Key Vault

Examples include:

API keys
Connection strings
Certificates

Managed Identities

Managed identities reduce the need for stored credentials.

Benefits:

Improved security
Simplified authentication
Reduced secret exposure

Role-Based Access Control (RBAC)

RBAC limits access to:

Deployments
Resources
Pipelines
Secrets

Monitoring CI/CD Pipelines

Pipelines should monitor:

Build failures
Deployment failures
Performance regressions
AI quality degradation

Azure Monitor

Azure Monitor supports:

Metrics
Alerts
Logging
Diagnostics

Application Insights

Application Insights helps monitor:

API latency
Failures
Dependency performance
User behavior

AI-Specific Monitoring

AI systems should monitor:

Token usage
Hallucination rates
Retrieval quality
Tool execution failures
Prompt performance

Common AI-103 CI/CD Scenarios

Scenario 1: Enterprise AI Copilot

Requirements:

Frequent prompt updates
Safe production releases
Automated testing

Recommended Approach:

GitHub Actions
Prompt Flow evaluations
Canary deployments

Scenario 2: Large-Scale AI Platform

Requirements:

Infrastructure automation
Multi-environment deployment
Enterprise governance

Recommended Approach:

Azure DevOps
Bicep or Terraform
Approval gates

Scenario 3: AI Agent Workflow System

Requirements:

Frequent workflow updates
Tool integration testing
Prompt validation

Recommended Approach:

Automated evaluation pipelines
Integration testing
Blue-green deployment strategy

Cost Optimization in CI/CD

CI/CD pipelines can increase operational costs.

Cost Optimization Strategies

Use Automated Cleanup

Remove:

Temporary environments
Test resources
Unused deployments

Optimize Test Frequency

Run expensive evaluations only when necessary.

Use Smaller Models for Testing

Smaller models reduce:

Token usage
Compute costs
Evaluation expenses

Common AI-103 Exam Tips

Understand CI/CD Fundamentals

Know:

Continuous Integration
Continuous Deployment
Automated testing
Deployment automation

Learn Deployment Strategies

Understand:

Blue-green deployments
Canary deployments
Rolling deployments
Rollback strategies

Know Infrastructure-as-Code Concepts

Understand:

Bicep
Terraform
ARM templates

Understand AI-Specific Testing

AI systems require testing for:

Prompt quality
Groundedness
Safety
Retrieval accuracy
Latency

Summary

Integrating Azure AI Foundry projects with CI/CD pipelines enables organizations to:

Automate deployments
Improve reliability
Increase scalability
Reduce operational risk
Accelerate AI delivery

For the AI-103 exam, you should understand:

CI/CD fundamentals
Azure DevOps pipelines
GitHub Actions workflows
Infrastructure-as-Code
Automated AI deployment strategies
Environment management
AI testing pipelines
Monitoring and observability
Secure deployment practices
Rollback and release strategies

Strong CI/CD practices are essential for building production-grade AI applications and agent-based systems on Azure.

Practice Exam Questions

Question 1

What does CI/CD stand for?

A. Continuous Integration and Continuous Deployment
B. Centralized Integration and Continuous Diagnostics
C. Continuous Inspection and Cloud Deployment
D. Centralized Infrastructure and Cloud Distribution

Answer

A. Continuous Integration and Continuous Deployment

Explanation

CI/CD automates software and AI deployment workflows.

Question 2

Which Azure service is commonly used for enterprise CI/CD pipelines?

A. Azure DevOps
B. Azure Backup
C. Azure DNS
D. Azure Files

Answer

A. Azure DevOps

Explanation

Azure DevOps provides build, release, and deployment pipeline capabilities.

Question 3

Which GitHub feature supports automated workflow execution for deployments?

A. GitHub Actions
B. GitHub Storage
C. GitHub Search
D. GitHub Monitor

Answer

A. GitHub Actions

Explanation

GitHub Actions automates workflows, testing, and deployments.

Question 4

Which deployment strategy uses two environments and switches traffic after validation?

A. Rolling deployment
B. Blue-green deployment
C. Canary deployment
D. Manual deployment

Answer

B. Blue-green deployment

Explanation

Blue-green deployments reduce downtime and simplify rollback.

Question 5

Which Azure service securely stores secrets for CI/CD pipelines?

A. Azure Key Vault
B. Azure Monitor
C. Azure Firewall
D. Azure CDN

Answer

A. Azure Key Vault

Explanation

Azure Key Vault securely stores secrets and credentials.

Question 6

Which Infrastructure-as-Code language is specifically designed for Azure?

A. Bicep
B. SQL
C. JavaScript
D. HTML

Answer

A. Bicep

Explanation

Bicep is a declarative Infrastructure-as-Code language for Azure.

Question 7

What is the primary purpose of canary deployments?

A. Eliminate monitoring
B. Gradually release updates to reduce risk
C. Replace version control
D. Encrypt model endpoints

Answer

B. Gradually release updates to reduce risk

Explanation

Canary deployments expose updates to a subset of users first.

Question 8

Which type of testing validates interactions between models, APIs, and databases?

A. Unit testing
B. Integration testing
C. Syntax testing
D. Deployment testing

Answer

B. Integration testing

Explanation

Integration testing validates component interactions.

Question 9

Which Azure service helps monitor application telemetry and diagnostics?

A. Application Insights
B. Azure DNS
C. Azure Backup
D. Azure Files

Answer

A. Application Insights

Explanation

Application Insights provides telemetry and monitoring capabilities.

Question 10

Which Azure feature reduces the need to store credentials directly in pipelines?

A. Managed identities
B. Public IP addresses
C. Azure CDN
D. Static tokens

Answer

A. Managed identities

Explanation

Managed identities provide secure authentication without storing credentials.

Go to the AI-103 Exam Prep Hub main page

AI, AI-103, Artificial Intelligence (AI), Microsoft Certification May 25, 2026

Configure model and agent deployments (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Set up AI solutions in Foundry
      --> Configure model and agent deployments

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

One of the most important responsibilities for Azure AI developers is configuring and managing model and agent deployments.

Modern AI applications depend on properly configured:

Large Language Models (LLMs)
Embedding models
Multimodal models
AI agents
Retrieval systems
Tool integrations
Orchestration workflows

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your ability to configure AI solutions in Azure AI Foundry and related Azure services.

For the AI-103 exam, you should understand:

Azure OpenAI model deployments
Deployment types
Provisioned throughput
Model versioning
Deployment scaling
Agent configuration
Tool and function integration
Retrieval integration
Security configuration
Monitoring and evaluation
Deployment lifecycle management

What Is a Model Deployment?

A model deployment is a configured instance of an AI model that applications can access through APIs.

Deployments allow developers to:

Choose models
Configure capacity
Control scaling
Manage versions
Apply security controls
Monitor usage

A deployment acts as the operational endpoint for AI inference.

Azure AI Foundry

Azure AI Foundry provides tools and services for:

Deploying AI models
Configuring AI agents
Managing workflows
Evaluating AI systems
Monitoring AI applications

It integrates with:

Azure OpenAI
Azure AI Search
Prompt Flow
Azure AI Content Safety
Azure Functions

Types of Models in Azure AI

Common model types include:

Large Language Models (LLMs)
Small Language Models (SLMs)
Embedding models
Multimodal models
Vision models
Speech models

Large Language Models (LLMs)

LLMs are used for:

Chatbots
AI copilots
Summarization
Reasoning
Tool calling
Content generation

Examples include GPT-based models.

Embedding Models

Embedding models convert content into vector representations.

Used for:

Vector search
Semantic retrieval
Similarity matching
RAG systems

Multimodal Models

Multimodal models process multiple input types such as:

Text
Images
Audio
Documents

Used for:

Image analysis
Visual reasoning
OCR workflows
Multimodal agents

Azure OpenAI Deployments

Azure OpenAI deployments expose models through API endpoints.

Deployment configuration includes:

Model selection
Deployment name
Capacity allocation
Version selection
Region selection
Content filtering settings

Deployment Names

Each deployment has a unique deployment name.

Applications use the deployment name when making API requests.

Example:

gpt4-copilot-prod
embeddings-search-dev

Model Versioning

Models evolve over time.

Versioning helps:

Maintain stability
Test upgrades
Support rollback strategies
Compare model behavior

Why Model Versioning Matters

Different versions may:

Behave differently
Produce different outputs
Affect latency
Affect costs
Impact prompt performance

Deployment Types

Azure AI commonly supports:

Standard deployments
Provisioned throughput deployments

Standard Deployments

Standard deployments use shared infrastructure.

Advantages:

Simpler setup
Lower upfront costs
Flexible usage

Limitations:

Shared capacity
Variable latency under heavy load

Provisioned Throughput Deployments

Provisioned throughput reserves dedicated model capacity.

Advantages:

Predictable performance
Consistent latency
Enterprise-grade scaling

Limitations:

Higher cost
Capacity planning required

When to Use Standard Deployments

Use standard deployments when:

Workloads are moderate
Usage is variable
Cost optimization matters
Development/testing environments are used

When to Use Provisioned Throughput

Use provisioned throughput when:

High traffic is expected
Predictable latency is required
Enterprise SLAs exist
Production copilots are deployed

Scaling Model Deployments

AI deployments must support varying workloads.

Autoscaling

Autoscaling adjusts resources dynamically based on demand.

Benefits:

Improved performance
Better cost efficiency
Reduced manual intervention

Horizontal Scaling

Horizontal scaling adds additional instances or capacity.

Useful for:

High concurrency
Enterprise AI systems
Large-scale chatbots

Latency Considerations

Latency refers to response time.

Factors affecting latency:

Model size
Throughput load
Geographic distance
Retrieval pipelines
Tool execution

Choosing the Correct Model

Choosing the correct model is critical.

Use Larger Models When:

Advanced reasoning is required
Complex workflows exist
High-quality generation matters

Use Smaller Models When:

Cost efficiency matters
Low latency is important
Simpler tasks are performed

Agent Deployments

AI agents combine:

Models
Memory
Retrieval
Tool calling
Workflow orchestration

Agent deployment involves configuring all these components together.

Agent Configuration Components

Common agent configuration elements include:

System prompts
Tool definitions
Function calling
Knowledge sources
Retrieval settings
Memory configuration
Safety settings

System Prompts

System prompts define:

Agent behavior
Role instructions
Response style
Operational constraints

Well-designed system prompts improve:

Reliability
Consistency
Safety

Tool and Function Integration

Agents may use tools such as:

APIs
Databases
Search services
External systems

Function calling enables agents to invoke these tools dynamically.

Retrieval Integration

Many AI agents use Retrieval-Augmented Generation (RAG).

RAG systems commonly integrate:

Azure AI Search
Embedding models
Vector search
Knowledge indexes

Knowledge Sources

Agents may connect to:

Enterprise documents
Databases
APIs
SharePoint
Blob Storage
Internal knowledge bases

Memory Configuration

Agents may use:

Short-term memory
Long-term memory
Semantic memory

Common storage systems include:

Azure Cosmos DB
Azure SQL Database
Azure AI Search

Security Configuration

Security is a major AI-103 exam topic.

Microsoft Entra ID

Microsoft Entra ID supports:

Authentication
Authorization
RBAC
Identity management

Azure Key Vault

Azure Key Vault securely stores:

API keys
Secrets
Certificates
Connection strings

Content Safety Configuration

Azure AI Content Safety helps:

Detect harmful content
Filter unsafe outputs
Apply safety policies

Network Security

Enterprise AI deployments may use:

VNets
Private Endpoints
Firewalls
API gateways

Monitoring Deployments

AI deployments require operational monitoring.

Azure Monitor

Azure Monitor provides:

Metrics
Logging
Alerts
Diagnostics

Application Insights

Application Insights supports:

Telemetry
Request tracing
Error diagnostics
Performance monitoring

Metrics to Monitor

Common metrics include:

Latency
Token usage
Error rates
Throughput
Tool call failures
Retrieval quality

Evaluating AI Deployments

AI systems should be evaluated for:

Accuracy
Groundedness
Safety
Relevance
Reliability

Prompt Flow

Prompt Flow supports:

Workflow orchestration
Prompt chaining
Tool integration
Evaluation pipelines

Prompt Flow is an important AI-103 topic.

CI/CD for AI Deployments

AI deployment pipelines should support:

Automated testing
Version control
Safe releases
Rollbacks

Blue-Green Deployments

Blue-green deployments:

Reduce downtime
Support safer releases
Simplify rollback

Canary Deployments

Canary deployments:

Roll out changes gradually
Reduce deployment risk
Support controlled testing

Common AI-103 Deployment Scenarios

Scenario 1: Enterprise AI Copilot

Requirements:

High concurrency
Secure retrieval
Enterprise search
Low latency

Recommended Configuration:

Provisioned throughput
Azure AI Search
Entra ID
Autoscaling

Scenario 2: Development Chatbot

Requirements:

Low cost
Rapid experimentation
Flexible scaling

Recommended Configuration:

Standard deployment
App Service
Basic monitoring

Scenario 3: AI Agent with Tool Calling

Requirements:

API integrations
Workflow execution
Multi-step reasoning

Recommended Configuration:

Azure OpenAI
Azure Functions
Prompt Flow
Tool definitions

Scenario 4: Enterprise Knowledge Assistant

Requirements:

Grounded responses
Semantic retrieval
Document search

Recommended Configuration:

Embedding models
Azure AI Search
Hybrid search
RAG pipelines

Cost Optimization Considerations

AI deployments can become expensive.

Common Cost Drivers

Token usage
Provisioned throughput
Search indexing
Embedding generation
Large models
High concurrency

Cost Optimization Strategies

Use Smaller Models When Possible

Smaller models reduce:

Latency
Compute costs
Token usage

Optimize Retrieval

Efficient retrieval reduces:

Prompt size
Token costs
Latency

Use Autoscaling

Autoscaling prevents overprovisioning.

Common AI-103 Exam Tips

Understand Deployment Types

Know the differences between:

Standard deployments
Provisioned throughput deployments

Learn Agent Configuration Components

Understand:

System prompts
Tool integration
Retrieval settings
Memory configuration

Know Security Best Practices

Use:

Entra ID
RBAC
Key Vault
Private networking

Understand Monitoring Concepts

Know how to monitor:

Latency
Token usage
Throughput
Errors
AI quality

Summary

Configuring model and agent deployments is a critical skill for Azure AI developers.

For the AI-103 exam, you should understand:

Azure OpenAI deployment configuration
Model versioning
Deployment scaling
Agent architecture
Tool integration
Retrieval integration
Memory configuration
Security controls
Monitoring and evaluation
Deployment lifecycle management

Well-configured deployments improve:

Reliability
Performance
Scalability
Security
Cost efficiency
User experience

These concepts are foundational for building enterprise-grade AI applications and agent-based systems on Azure.

Practice Exam Questions

Question 1

Which deployment type provides dedicated capacity for Azure OpenAI workloads?

A. Shared deployment
B. Provisioned throughput deployment
C. Batch deployment
D. Basic deployment

Answer

B. Provisioned throughput deployment

Explanation

Provisioned throughput reserves dedicated processing capacity.

Question 2

What is the primary purpose of model versioning?

A. Increase storage size
B. Manage model updates and rollback strategies
C. Reduce API authentication
D. Eliminate monitoring

Answer

B. Manage model updates and rollback strategies

Explanation

Versioning helps maintain stability and supports rollback.

Question 3

Which Azure service is MOST commonly used for semantic retrieval in RAG systems?

A. Azure AI Search
B. Azure Backup
C. Azure CDN
D. Azure DNS

Answer

A. Azure AI Search

Explanation

Azure AI Search supports vector and semantic retrieval.

Question 4

What is the purpose of a system prompt in an AI agent?

A. Encrypt embeddings
B. Define agent behavior and instructions
C. Replace APIs
D. Configure storage replication

Answer

B. Define agent behavior and instructions

Explanation

System prompts guide the agent’s role, constraints, and response style.

Question 5

Which Azure service securely stores API keys and secrets?

A. Azure Key Vault
B. Azure Monitor
C. Azure Backup
D. Azure CDN

Answer

A. Azure Key Vault

Explanation

Azure Key Vault securely stores sensitive credentials.

Question 6

Which deployment strategy gradually rolls out updates to a small percentage of users first?

A. Full deployment
B. Canary deployment
C. Offline deployment
D. Batch deployment

Answer

B. Canary deployment

Explanation

Canary deployments reduce deployment risk through gradual rollout.

Question 7

Which type of model is specifically designed for vector generation and semantic similarity?

A. Vision model
B. Embedding model
C. Speech model
D. OCR model

Answer

B. Embedding model

Explanation

Embedding models generate vector representations for semantic retrieval.

Question 8

Which Azure service provides telemetry and request tracing for AI applications?

A. Application Insights
B. Azure DNS
C. Azure Files
D. Azure Firewall

Answer

A. Application Insights

Explanation

Application Insights provides application telemetry and diagnostics.

Question 9

Which feature dynamically adjusts resources based on workload demand?

A. Static allocation
B. Autoscaling
C. Encryption scaling
D. Semantic routing

Answer

B. Autoscaling

Explanation

Autoscaling automatically adjusts capacity based on traffic.

Question 10

Which Azure service is commonly used for workflow orchestration and prompt chaining in AI solutions?

A. Prompt Flow
B. Azure CDN
C. Azure Backup
D. Azure Front Door

Answer

A. Prompt Flow

Explanation

Prompt Flow orchestrates prompts, tools, and AI workflows.

Go to the AI-103 Exam Prep Hub main page

AI, AI-103, Artificial Intelligence (AI), Microsoft Certification May 25, 2026

Design Azure infrastructure for AI Apps and agent-based solutions (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Set up AI solutions in Foundry
      --> Design Azure infrastructure for AI Apps and agent-based solutions

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Designing infrastructure for AI applications and agent-based systems is one of the most important responsibilities for Azure AI developers.

Modern AI solutions are not simply standalone models. They are distributed cloud systems that combine:

AI services
APIs
Databases
Search systems
Storage
Networking
Security controls
Monitoring systems
Agent orchestration components

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your ability to design Azure infrastructure that supports:

Generative AI applications
AI agents
Retrieval-Augmented Generation (RAG)
Vector search
Multimodal AI systems
Scalable AI architectures
Secure enterprise AI deployments

For the AI-103 exam, you should understand:

Core Azure infrastructure services
AI architecture patterns
Scalability and performance design
Networking and security
Identity and access management
Storage and databases
Monitoring and observability
Cost optimization
High availability and disaster recovery
Infrastructure choices for AI agents

Core Components of AI Infrastructure

AI applications commonly require multiple infrastructure layers.

Typical components include:

AI model services
Compute resources
Storage systems
Search and retrieval systems
Networking components
Security services
Monitoring systems
Workflow orchestration
API management
Identity management

Azure AI Services Layer

Azure OpenAI

Azure OpenAI provides:

Large Language Models (LLMs)
Embedding models
Multimodal models
Conversational AI capabilities

Azure OpenAI is commonly used for:

AI copilots
Chatbots
AI agents
Summarization
Content generation
Tool calling

Azure AI Search

Azure AI Search supports:

Vector search
Semantic search
Hybrid search
Enterprise retrieval
RAG architectures

It is commonly used for:

Knowledge grounding
Enterprise search
AI assistant retrieval

Azure AI Vision

Azure AI Vision provides:

OCR
Image analysis
Object detection
Caption generation
Visual understanding

Azure AI Document Intelligence

Azure AI Document Intelligence supports:

Invoice extraction
Form processing
Layout analysis
OCR workflows
Structured document extraction

Compute Infrastructure for AI Applications

Azure App Service

Azure App Service is commonly used to host:

Web applications
AI front ends
APIs
Lightweight AI services

Advantages:

Managed platform
Easy scaling
Simplified deployment

Azure Kubernetes Service (AKS)

AKS provides container orchestration for:

Large-scale AI applications
Microservices
Agent orchestration systems
Distributed AI workloads

Advantages:

High scalability
Container management
Advanced orchestration
Enterprise-grade deployments

When to Use AKS

Use AKS when:

Complex orchestration is required
Multiple services interact
High scalability is needed
Microservice architectures are used

Azure Functions

Azure Functions provides serverless compute.

Common AI use cases:

Tool execution
Event-driven workflows
API integrations
Lightweight processing
Agent tool calling

Advantages:

Pay-per-use pricing
Automatic scaling
Fast development

Azure Container Apps

Azure Container Apps provides simplified container hosting.

Useful for:

API services
AI middleware
Lightweight agent services
Event-driven AI components

Choosing the Correct Compute Service

Use Azure App Service When:

Hosting simple AI web apps
Managing APIs
Rapid deployment is needed

Use AKS When:

Large-scale orchestration is required
Complex microservices exist
Advanced scalability is necessary

Use Azure Functions When:

Event-driven execution is needed
Tool calling is required
Lightweight compute is sufficient

Use Azure Container Apps When:

Container simplicity is preferred
Serverless containers are desired

Storage Infrastructure

AI systems often require multiple storage solutions.

Azure Blob Storage

Azure Blob Storage supports:

Document storage
Training data
Images
Videos
Logs
AI datasets

Common AI uses:

RAG document storage
Knowledge repositories
Media storage

Azure Cosmos DB

Azure Cosmos DB provides:

Globally distributed NoSQL storage
Low-latency access
High scalability

Common AI uses:

Agent memory
Session storage
User profiles
Conversation history

Azure SQL Database

Azure SQL Database supports:

Structured enterprise data
Relational workloads
Transactional systems

Common AI uses:

Enterprise integration
Business systems
Structured metadata

Vector Storage

Vector-enabled storage supports:

Embedding storage
Similarity search
Semantic retrieval

Common services include:

Azure AI Search
Azure Cosmos DB
Azure SQL Database

Networking Infrastructure

AI solutions require secure and scalable networking.

Virtual Networks (VNets)

VNets provide:

Network isolation
Secure communication
Private connectivity

Use VNets when:

Enterprise security is required
Private networking is necessary
Sensitive data is involved

Private Endpoints

Private Endpoints allow Azure services to be accessed privately through VNets.

Benefits:

Improved security
Reduced public exposure
Enterprise compliance support

API Management

Azure API Management helps:

Secure APIs
Throttle requests
Monitor API usage
Apply policies
Manage agent APIs

This is important for:

AI agents
Tool integrations
Enterprise API governance

Load Balancing

Azure Load Balancer and Application Gateway help:

Distribute traffic
Improve availability
Scale AI applications

Identity and Security

Security is a major AI-103 exam topic.

Microsoft Entra ID

Microsoft Entra ID provides:

Authentication
Authorization
Identity management
Role-based access control (RBAC)

AI applications use Entra ID for:

User authentication
API access control
Secure enterprise integration

Role-Based Access Control (RBAC)

RBAC ensures users and services only access authorized resources.

Examples:

Restricting AI model access
Controlling storage access
Securing search indexes

Azure Key Vault

Azure Key Vault stores:

Secrets
API keys
Certificates
Connection strings

Never hardcode secrets in AI applications.

Azure AI Content Safety

Azure AI Content Safety helps:

Detect harmful content
Filter unsafe outputs
Support responsible AI practices

Monitoring and Observability

AI systems require monitoring for:

Reliability
Performance
Cost
Failures
Hallucinations
API latency

Azure Monitor

Azure Monitor collects:

Metrics
Logs
Alerts
Performance data

Application Insights

Application Insights supports:

Application telemetry
Request tracing
Error tracking
Dependency monitoring

Useful for:

AI apps
APIs
Agent workflows

Logging AI Systems

AI systems should log:

Prompts
Responses
Errors
Tool calls
Latency
Retrieval quality

Logging helps:

Troubleshooting
Auditing
Evaluation
Compliance

Scalability Design

AI applications may experience:

High traffic
Large token volumes
Heavy retrieval workloads
Concurrent agent operations

Infrastructure must scale effectively.

Horizontal Scaling

Horizontal scaling adds more instances.

Examples:

Additional API servers
More containers
More worker nodes

Vertical Scaling

Vertical scaling increases resource capacity.

Examples:

More CPU
More memory
Larger VM sizes

Autoscaling

Autoscaling dynamically adjusts resources based on demand.

Common services supporting autoscaling:

AKS
Azure Functions
App Service
Container Apps

High Availability and Disaster Recovery

Enterprise AI systems require resilience.

Availability Zones

Availability Zones improve fault tolerance.

Benefits:

Redundancy
Improved uptime
Reduced outage risk

Geo-Redundancy

Geo-redundancy replicates data across regions.

Useful for:

Disaster recovery
Business continuity
Global applications

Backup and Recovery

AI systems should back up:

Knowledge indexes
Databases
Configuration data
Logs
Agent memory

Infrastructure for AI Agents

AI agents often require additional infrastructure components.

Agent Orchestration

AI agents may require orchestration services such as:

Prompt Flow
Azure Functions
Logic Apps
AKS workflows

Retrieval Infrastructure

Agent systems commonly use:

Azure AI Search
Embeddings
Vector indexes
RAG pipelines

Persistent Memory Infrastructure

Persistent memory may use:

Azure Cosmos DB
Azure SQL Database
Blob Storage

Tool Integration Infrastructure

Agents often integrate with:

REST APIs
Databases
External SaaS systems
Enterprise workflows

Common AI-103 Architecture Scenarios

Scenario 1: Enterprise AI Copilot

Requirements:

Conversational AI
Enterprise search
Secure authentication
Document retrieval

Recommended Infrastructure:

Azure OpenAI
Azure AI Search
Entra ID
Blob Storage
App Service

Scenario 2: Large-Scale Multi-Agent System

Requirements:

Multiple AI agents
High scalability
Distributed orchestration

Recommended Infrastructure:

AKS
Azure Functions
Prompt Flow
Cosmos DB

Scenario 3: AI Invoice Processing Solution

Requirements:

OCR
Document extraction
Workflow automation

Recommended Infrastructure:

Azure AI Document Intelligence
Blob Storage
Logic Apps
Azure Functions

Scenario 4: Global AI Chat Platform

Requirements:

Global availability
High concurrency
Disaster recovery

Recommended Infrastructure:

Geo-redundant storage
Availability Zones
Load balancing
Autoscaling

Cost Optimization Considerations

AI infrastructure can become expensive.

Common Cost Drivers

Token usage
Vector storage
GPU workloads
Data transfer
Search indexing
High-scale orchestration

Cost Optimization Strategies

Use Smaller Models When Appropriate

Smaller models reduce:

Compute usage
Token costs
Latency

Use Autoscaling

Autoscaling reduces idle resource costs.

Optimize Retrieval Pipelines

Efficient chunking and indexing reduce:

Search costs
Storage requirements
Retrieval latency

Common AI-103 Exam Tips

Understand Infrastructure Tradeoffs

Know when to use:

AKS vs App Service
Functions vs Containers
Cosmos DB vs SQL Database

Learn Security Best Practices

Know how to use:

Entra ID
RBAC
Key Vault
Private Endpoints

Understand RAG Infrastructure

RAG commonly uses:

Azure OpenAI
Azure AI Search
Embeddings
Storage systems

Know Agent Infrastructure Patterns

AI agents commonly require:

Workflow orchestration
Tool integration
Persistent memory
Retrieval systems

Summary

Designing Azure infrastructure for AI applications requires balancing:

Scalability
Security
Performance
Cost
Reliability
Maintainability

For the AI-103 exam, you should understand:

Azure AI service architecture
Compute options
Storage design
Networking and security
Monitoring and observability
High availability
Agent infrastructure patterns
RAG infrastructure
Infrastructure scaling strategies

Strong infrastructure design skills are essential for deploying production-grade AI apps and agent-based systems on Azure.

Practice Exam Questions

Question 1

Which Azure service is MOST appropriate for enterprise vector search and RAG retrieval?

A. Azure AI Search
B. Azure Backup
C. Azure CDN
D. Azure DNS

Answer

A. Azure AI Search

Explanation

Azure AI Search supports vector search, semantic search, and retrieval for RAG systems.

Question 2

Which Azure compute service is BEST suited for large-scale containerized AI microservices?

A. Azure App Service
B. Azure Kubernetes Service (AKS)
C. Azure Files
D. Azure CDN

Answer

B. Azure Kubernetes Service (AKS)

Explanation

AKS provides advanced container orchestration and scalability.

Question 3

Which Azure service is MOST appropriate for storing API keys and secrets securely?

A. Azure Key Vault
B. Azure Monitor
C. Azure DNS
D. Azure Load Balancer

Answer

A. Azure Key Vault

Explanation

Azure Key Vault securely stores secrets, certificates, and keys.

Question 4

Which Azure service provides serverless execution for lightweight AI workflows and tool calling?

A. Azure Functions
B. Azure Backup
C. Azure CDN
D. Azure Firewall

Answer

A. Azure Functions

Explanation

Azure Functions supports event-driven serverless compute.

Question 5

What is the primary purpose of Availability Zones?

A. Reduce token usage
B. Improve fault tolerance and uptime
C. Replace backups
D. Encrypt embeddings

Answer

B. Improve fault tolerance and uptime

Explanation

Availability Zones provide redundancy across isolated datacenter locations.

Question 6

Which Azure service is MOST commonly used for globally distributed NoSQL storage in AI applications?

A. Azure Cosmos DB
B. Azure DNS
C. Azure Files
D. Azure CDN

Answer

A. Azure Cosmos DB

Explanation

Azure Cosmos DB provides scalable globally distributed NoSQL storage.

Question 7

Which Azure networking feature enables private access to Azure services from a VNet?

A. Private Endpoint
B. Public IP
C. Load Balancer
D. Traffic Manager

Answer

A. Private Endpoint

Explanation

Private Endpoints provide secure private connectivity.

Question 8

Which Azure monitoring service provides application telemetry and request tracing?

A. Application Insights
B. Azure CDN
C. Azure Policy
D. Azure ExpressRoute

Answer

A. Application Insights

Explanation

Application Insights provides telemetry and diagnostics for applications.

Question 9

Which Azure identity service provides authentication and RBAC support for AI applications?

A. Microsoft Entra ID
B. Azure CDN
C. Azure Firewall
D. Azure Front Door

Answer

A. Microsoft Entra ID

Explanation

Microsoft Entra ID provides identity and access management.

Question 10

Which scaling strategy adds additional instances to support increased AI workload demand?

A. Vertical scaling
B. Horizontal scaling
C. Encryption scaling
D. Semantic scaling

Answer

B. Horizontal scaling

Explanation

Horizontal scaling adds more instances to distribute workloads.

Go to the AI-103 Exam Prep Hub main page

AI, AI-103, Artificial Intelligence (AI), Microsoft Certification May 25, 2026

Choose appropriate memory, tool, and knowledge integration services for agent solutions (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Choose the appropriate Foundry services for generative AI and agents
      --> Choose appropriate memory, tool, and knowledge integration services for agent solutions

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI agents are far more advanced than traditional chatbots.

AI agents can:

Reason through problems
Plan tasks
Access tools
Retrieve knowledge
Maintain conversational memory
Execute workflows
Interact with enterprise systems
Coordinate multi-step operations

The AI-103: Develop AI Apps and Agents on Azure certification exam places significant emphasis on understanding how to design and implement these agent capabilities using Azure AI Foundry and related Azure services.

One of the most important skills tested on the exam is the ability to choose appropriate:

Memory systems
Tool integration services
Knowledge integration services
Retrieval architectures
Agent orchestration tools

For the AI-103 exam, you should understand:

Different types of agent memory
Tool calling and function calling
Retrieval-Augmented Generation (RAG)
Knowledge grounding
Azure AI Search integration
Agent orchestration workflows
External API integration
Vector search and embeddings
Enterprise knowledge integration
Security and governance considerations

What Are AI Agents?

AI agents are AI-powered systems capable of:

Interpreting goals
Planning actions
Using tools
Retrieving information
Maintaining context
Completing tasks autonomously or semi-autonomously

Unlike traditional chatbots, AI agents can:

Interact with APIs
Execute workflows
Use memory
Retrieve enterprise knowledge
Chain actions together
Adapt dynamically to user requests

Components of an AI Agent Architecture

Modern AI agent solutions commonly include:

Large Language Models (LLMs)
Memory systems
Retrieval systems
Knowledge integration
Tool and function calling
Workflow orchestration
Security and governance controls

Azure AI Foundry and Agent Solutions

Azure AI Foundry provides services and tools that help developers:

Build AI agents
Integrate tools
Connect enterprise knowledge
Implement RAG
Orchestrate workflows
Evaluate agent behavior
Monitor AI systems

Core services often include:

Azure OpenAI
Azure AI Search
Prompt Flow
Azure AI Content Safety
Azure Functions
Azure Logic Apps
Azure Cosmos DB
Azure SQL Database

Memory in AI Agents

What Is Agent Memory?

Memory enables AI agents to retain and use information over time.

Memory allows agents to:

Maintain conversational context
Remember user preferences
Track workflow state
Store historical interactions
Support long-running tasks

Without memory, every interaction becomes isolated.

Types of Agent Memory

The AI-103 exam may test multiple memory types.

Short-Term Memory

What Is Short-Term Memory?

Short-term memory stores temporary conversational context.

Examples:

Current chat history
Active task context
Immediate instructions

Characteristics of Short-Term Memory

Session-based
Temporary
Fast access
Often stored in prompts or session state

When to Use Short-Term Memory

Use short-term memory for:

Conversational continuity
Current workflow tracking
Multi-turn conversations

Long-Term Memory

What Is Long-Term Memory?

Long-term memory stores persistent information across sessions.

Examples:

User preferences
Historical interactions
Persistent profiles
Prior decisions

Characteristics of Long-Term Memory

Persistent storage
Cross-session continuity
Larger storage capacity
Supports personalization

Azure Services for Long-Term Memory

Common services include:

Azure Cosmos DB
Azure SQL Database
Azure Storage
Vector databases

When to Use Long-Term Memory

Use long-term memory when:

Personalization is required
User preferences must persist
Historical context matters
Long-running workflows exist

Semantic Memory

What Is Semantic Memory?

Semantic memory stores knowledge in embeddings or vectorized formats.

This enables:

Semantic retrieval
Knowledge recall
Contextual understanding
Similarity matching

Semantic Memory in AI Agents

Semantic memory often uses:

Embedding models
Vector search
Azure AI Search

This allows agents to retrieve relevant information dynamically.

Episodic Memory

What Is Episodic Memory?

Episodic memory stores records of past interactions and events.

Examples:

Past conversations
Completed workflows
User activity history

This helps agents maintain continuity across interactions.

Choosing the Correct Memory Type

Use Short-Term Memory When:

Managing active conversations
Maintaining immediate context
Supporting temporary tasks

Use Long-Term Memory When:

Storing persistent user information
Personalizing experiences
Maintaining history across sessions

Use Semantic Memory When:

Retrieving knowledge semantically
Supporting RAG
Performing contextual retrieval

Use Episodic Memory When:

Tracking prior interactions
Supporting historical continuity

Knowledge Integration

What Is Knowledge Integration?

Knowledge integration connects AI agents to external information sources.

Examples:

Enterprise documents
Databases
Knowledge bases
APIs
Websites
Internal systems

Knowledge integration helps agents:

Provide grounded answers
Access current information
Reduce hallucinations
Support enterprise use cases

Retrieval-Augmented Generation (RAG)

What Is RAG?

RAG combines:

Retrieval systems
Search indexes
Embeddings
LLMs

RAG enables agents to retrieve external information before generating responses.

Azure AI Search for Knowledge Integration

Azure AI Search is a core service for:

Vector search
Semantic search
Hybrid search
Enterprise retrieval
Knowledge grounding

It enables agents to:

Search enterprise documents
Retrieve semantically relevant content
Access indexed knowledge

Hybrid Search

Hybrid search combines:

Keyword search
Semantic ranking
Vector search

Hybrid search is often the preferred approach for enterprise AI agents.

Embeddings and Knowledge Retrieval

Embedding models convert content into vector representations.

Embeddings support:

Semantic similarity
Vector retrieval
Knowledge recall
RAG pipelines

Azure OpenAI embedding models are commonly used.

Knowledge Sources for AI Agents

AI agents may integrate with:

Azure Blob Storage
SharePoint
Databases
REST APIs
Enterprise document repositories
CRM systems
ERP systems

Tool Integration

What Is Tool Integration?

Tool integration enables AI agents to interact with external systems.

Examples include:

APIs
Databases
Email systems
Calendars
Search services
Workflow systems

Tool integration allows agents to perform actions instead of only generating text.

Tool Calling and Function Calling

LLMs can invoke:

Tools
Functions
APIs

Examples:

Retrieve weather data
Send emails
Query databases
Create support tickets
Execute workflows

Azure Services for Tool Integration

Common services include:

Azure Functions
Azure Logic Apps
REST APIs
Azure API Management

Azure Functions

Azure Functions provides serverless compute for:

API integrations
Business logic
Event-driven workflows
Tool execution

AI agents often call Azure Functions to execute tasks.

Azure Logic Apps

Azure Logic Apps supports:

Workflow automation
Enterprise integrations
Connector-based orchestration

Logic Apps are useful when:

Multiple systems must interact
Low-code orchestration is preferred
Enterprise automation is needed

Azure API Management

Azure API Management helps:

Secure APIs
Manage API access
Monitor API usage
Apply governance policies

Useful for enterprise AI agent integrations.

Prompt Flow

Prompt Flow is a Foundry tool for:

Building AI workflows
Orchestrating prompts
Chaining tools
Managing agent pipelines
Evaluating workflows

Prompt Flow is a major AI-103 exam topic.

Multi-Agent Systems

Some AI architectures use multiple specialized agents.

Examples:

Research agent
Scheduling agent
Data retrieval agent
Customer service agent

Multi-agent systems may improve:

Scalability
Specialization
Workflow separation

Orchestration Services

Agent orchestration coordinates:

Memory
Retrieval
Tool execution
Workflow management

Common orchestration tools include:

Prompt Flow
Azure Functions
Logic Apps
Custom orchestration frameworks

Security and Governance

AI agent systems require:

Authentication
Authorization
Data protection
Content filtering
Responsible AI controls

Azure AI Content Safety

Azure AI Content Safety helps:

Detect harmful content
Prevent unsafe outputs
Support responsible AI deployments

Role-Based Access Control (RBAC)

RBAC ensures agents only access authorized resources.

This is especially important for:

Enterprise knowledge systems
Confidential data
Regulated environments

Monitoring and Observability

AI agent systems should monitor:

Tool usage
Latency
Errors
Retrieval quality
Hallucinations
Token usage

Monitoring improves:

Reliability
Performance
Troubleshooting

Common AI-103 Scenarios

Scenario 1: Enterprise Copilot

Requirements:

Access enterprise documents
Remember user preferences
Retrieve current information
Support conversational interactions

Recommended Services:

Azure OpenAI
Azure AI Search
Embedding models
Long-term memory storage

Scenario 2: AI Travel Assistant

Requirements:

Access calendars
Book hotels
Query APIs
Manage workflows

Recommended Services:

Azure OpenAI
Tool/function calling
Azure Functions
Prompt Flow

Scenario 3: Customer Support Agent

Requirements:

Retrieve support documents
Track prior interactions
Escalate tickets

Recommended Services:

Azure AI Search
Episodic memory
Azure Functions
CRM integration

Scenario 4: Personalized Learning Assistant

Requirements:

Remember learning preferences
Track progress
Recommend materials

Recommended Services:

Long-term memory
Semantic retrieval
Azure Cosmos DB

Common AI-103 Exam Tips

Understand Memory Types

Know the differences between:

Short-term memory
Long-term memory
Semantic memory
Episodic memory

Know When to Use RAG

Use RAG when:

External knowledge is required
Current data is needed
Hallucination reduction matters

Learn Tool Calling Concepts

Agents use:

Function calling
APIs
Workflows
Tool orchestration

This is commonly tested.

Understand Azure Service Roles

Azure AI Search

Used for:

Retrieval
Vector search
Grounding

Azure Functions

Used for:

Executing logic
Tool integration

Prompt Flow

Used for:

Workflow orchestration
Agent pipelines

Azure Cosmos DB

Used for:

Persistent memory
Long-term storage

Summary

AI agents require more than just language models.

Successful agent solutions combine:

Memory systems
Retrieval systems
Knowledge grounding
Tool integration
Workflow orchestration
Security controls

For the AI-103 exam, you should understand:

Different memory architectures
Tool and function calling
RAG workflows
Azure AI Search integration
Knowledge retrieval strategies
Prompt Flow orchestration
Persistent memory services
Enterprise AI integration patterns

Understanding how these services work together is critical for building scalable and intelligent AI agent solutions.

Practice Exam Questions

Question 1

Which type of memory is MOST appropriate for maintaining conversational context during a single chat session?

A. Long-term memory
B. Semantic memory
C. Short-term memory
D. Episodic memory

Answer

C. Short-term memory

Explanation

Short-term memory maintains active conversational context within a session.

Question 2

Which Azure service is MOST commonly used for semantic retrieval and grounding in AI agents?

A. Azure AI Search
B. Azure Backup
C. Azure DNS
D. Azure Firewall

Answer

A. Azure AI Search

Explanation

Azure AI Search provides vector search and semantic retrieval capabilities.

Question 3

What is the primary purpose of Retrieval-Augmented Generation (RAG)?

A. Replace embeddings
B. Reduce retrieval latency only
C. Ground responses using retrieved information
D. Eliminate vector search

Answer

C. Ground responses using retrieved information

Explanation

RAG retrieves external information to improve groundedness and reduce hallucinations.

Question 4

Which Azure service is MOST appropriate for serverless tool execution within AI agents?

A. Azure Functions
B. Azure CDN
C. Azure Backup
D. Azure Policy

Answer

A. Azure Functions

Explanation

Azure Functions supports serverless execution of business logic and APIs.

Question 5

Which memory type stores knowledge using embeddings and vector representations?

A. Short-term memory
B. Semantic memory
C. Transactional memory
D. Procedural memory

Answer

B. Semantic memory

Explanation

Semantic memory stores information in vectorized forms for retrieval.

Question 6

Which Foundry tool is primarily used for orchestrating AI workflows and agent pipelines?

A. Azure Backup
B. Prompt Flow
C. Azure DNS
D. Azure Storage Explorer

Answer

B. Prompt Flow

Explanation

Prompt Flow supports workflow orchestration and prompt chaining.

Question 7

What is the primary advantage of long-term memory in AI agents?

A. Faster GPU performance
B. Persistent cross-session personalization
C. Lower token usage only
D. Reduced API calls

Answer

B. Persistent cross-session personalization

Explanation

Long-term memory enables persistent storage of preferences and history.

Question 8

Which Azure service is MOST appropriate for low-code workflow automation in enterprise agent systems?

A. Azure Logic Apps
B. Azure DNS
C. Azure Monitor
D. Azure DevTest Labs

Answer

A. Azure Logic Apps

Explanation

Azure Logic Apps provides low-code workflow orchestration and integrations.

Question 9

Which capability allows AI agents to invoke APIs and external systems dynamically?

A. OCR
B. Function calling
C. Metadata filtering
D. Image segmentation

Answer

B. Function calling

Explanation

Function calling enables AI models to interact with external tools and services.

Question 10

Which Azure service is MOST appropriate for persistent scalable storage of AI agent memory?

A. Azure Cosmos DB
B. Azure CDN
C. Azure Firewall
D. Azure ExpressRoute

Answer

A. Azure Cosmos DB

Explanation

Azure Cosmos DB is commonly used for scalable persistent memory storage.

Go to the AI-103 Exam Prep Hub main page