Category: Azure AI

Agentic AI, AI, AI-103, Azure AI, Generative AI, Microsoft Certification May 25, 2026

Implement model reflection, chain-of-thought evaluations, and self-critique loops (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Optimize and operationalize generative AI systems
      --> Implement model reflection, chain-of-thought evaluations, and self-critique loops

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

As generative AI systems become more advanced, developers increasingly need methods to improve reasoning quality, reduce hallucinations, increase reliability, and enhance agent decision-making. One of the most important areas in modern AI application design is implementing mechanisms that allow models to evaluate, refine, and improve their own outputs.

For the AI-103 certification exam, candidates must understand how to implement:

Model reflection
Chain-of-thought (CoT) evaluations
Self-critique loops
Iterative reasoning workflows
Verification and refinement strategies
Multi-step evaluation pipelines
Agent self-improvement mechanisms

These capabilities are especially important in:

AI agents
Retrieval-augmented generation (RAG)
Autonomous workflows
Multi-agent systems
Decision-support systems
Code generation systems
Enterprise copilots

This article explains the concepts, architectures, implementation strategies, Azure AI Foundry integration approaches, and best practices needed for the AI-103 exam.

Why Reflection and Self-Critique Matter

Large language models can generate impressive outputs, but they also have weaknesses:

Hallucinations
Logical inconsistencies
Missing steps
Incorrect assumptions
Unsafe outputs
Tool misuse
Incomplete reasoning
Weak grounding

Traditional prompting alone is often insufficient for enterprise-grade systems.

Reflection and critique techniques help models:

Re-evaluate outputs
Detect mistakes
Improve accuracy
Validate reasoning
Increase consistency
Improve grounding quality
Reduce unsafe behavior
Produce higher-confidence responses

These mechanisms are critical for building trustworthy AI systems.

Understanding Model Reflection

What Is Model Reflection?

Model reflection is the process in which an AI model evaluates its own output before returning a final response.

The model essentially asks itself:

Did I answer correctly?
Is my reasoning valid?
Did I follow instructions?
Is the answer grounded?
Is any information fabricated?
Is additional clarification needed?

Reflection can occur:

Internally during inference
As a separate evaluation pass
Through another model
Through an orchestrated pipeline
Inside an agent workflow

Reflection Workflow

A common reflection workflow includes:

User submits request
Model generates draft answer
Reflection stage evaluates output
Critique identifies weaknesses
Model revises answer
Final response returned

This creates an iterative improvement loop.

Types of Reflection

Single-Pass Reflection

The model reviews its response once before returning output.

Advantages:

Lower latency
Lower cost
Easier implementation

Disadvantages:

Limited correction depth
May miss subtle reasoning errors

Multi-Pass Reflection

The model repeatedly critiques and improves outputs.

Advantages:

Higher reasoning quality
Better correction capability
Improved reliability

Disadvantages:

Higher token consumption
Increased latency
More expensive

External Reflection

A second model evaluates the first model’s response.

Examples:

GPT-4 generates answer
Smaller evaluator model critiques answer
Safety model validates response
Grounding evaluator checks citations

Advantages:

Separation of generation and evaluation
Reduced bias
Specialized evaluators

Chain-of-Thought (CoT) Reasoning

What Is Chain-of-Thought?

Chain-of-thought prompting encourages the model to reason step-by-step instead of producing only a final answer.

Instead of:

“Answer this question.”

You might prompt:

“Think through the problem step-by-step before answering.”

This helps improve:

Mathematical reasoning
Logical analysis
Planning tasks
Multi-step decisions
Tool selection
Complex workflows

Benefits of Chain-of-Thought

Chain-of-thought reasoning helps:

Break problems into smaller steps
Reduce reasoning mistakes
Improve transparency
Enable debugging
Increase consistency
Improve agent planning

This is especially useful in:

AI agents
Financial analysis
Troubleshooting systems
Code generation
Workflow orchestration
Business reasoning

Example of Chain-of-Thought

Without Chain-of-Thought

Prompt:

“What is the total cost for 3 items priced at $20 each with 8% tax?”

Model output:

“$64.80”

With Chain-of-Thought

Prompt:

“Calculate the answer step-by-step.”

Model output:

3 items × $20 = $60
8% tax on $60 = $4.80
Total = $64.80

The reasoning becomes visible and easier to validate.

Chain-of-Thought Evaluations

What Are CoT Evaluations?

Chain-of-thought evaluations analyze the reasoning process itself rather than only the final answer.

The system evaluates:

Logical consistency
Step validity
Missing assumptions
Hallucinated reasoning
Unsupported claims
Unsafe logic

This is critical because a correct answer can still come from flawed reasoning.

Evaluating Reasoning Quality

Evaluation criteria may include:

Evaluation Area	Description
Accuracy	Is the final answer correct?
Logical Consistency	Are reasoning steps coherent?
Grounding	Is reasoning based on trusted data?
Completeness	Were all required steps included?
Safety	Did reasoning violate policy?
Hallucination Detection	Did the model invent facts?
Instruction Adherence	Did the model follow instructions?

Self-Critique Loops

What Is a Self-Critique Loop?

A self-critique loop is an iterative workflow in which the model:

Generates output
Critiques the output
Revises the output
Re-evaluates the revision
Produces a final response

This creates a feedback cycle.

Example Self-Critique Workflow

Step 1 — Initial Response

The model generates a draft answer.

Step 2 — Critique Prompt

The model receives instructions such as:

“Review your previous answer for factual inaccuracies, missing information, unsupported assumptions, or policy violations.”

Step 3 — Revision

The model revises the answer.

Step 4 — Final Validation

The system optionally performs:

Safety checks
Grounding checks
Relevance evaluation
Hallucination detection

Step 5 — Final Output

The improved answer is returned.

Benefits of Self-Critique Loops

Self-critique loops can:

Reduce hallucinations
Improve factual grounding
Improve code quality
Improve agent planning
Detect reasoning flaws
Increase answer completeness
Improve policy compliance
Reduce unsafe outputs

Reflection in Agentic Systems

Reflection is especially important in AI agents.

Agents often:

Use tools
Retrieve documents
Execute actions
Plan workflows
Make decisions
Coordinate multiple tasks

Without reflection, agents may:

Select incorrect tools
Misinterpret retrieved information
Perform unsafe actions
Produce incomplete workflows

Reflection helps agents verify:

Tool outputs
Action correctness
Goal completion
Reasoning quality
Constraint adherence

Reflection Architectures in Azure AI Foundry

Azure AI Foundry supports building reflection-enabled systems using:

Prompt flows
Agent orchestration
Evaluation pipelines
Safety evaluators
Retrieval pipelines
Tool calling
Monitoring systems

Common architecture components include:

Component	Purpose
LLM	Generates responses
Evaluator Model	Critiques outputs
Vector Search	Grounds responses
Prompt Flow	Orchestrates steps
Agent Memory	Stores conversation state
Safety Filters	Detect unsafe content
Monitoring Tools	Track quality metrics

Reflection Patterns

Generate → Critique → Revise

This is the most common pattern.

Flow:

Generate draft
Critique output
Revise response
Return final answer

Multi-Agent Reflection

One agent generates content while another agent critiques it.

Example:

Research agent gathers information
Reviewer agent checks accuracy
Compliance agent checks policy
Finalizer agent produces response

This improves specialization.

Debate Pattern

Two or more models debate possible answers.

Advantages:

Better reasoning exploration
Error detection
Stronger final conclusions

Disadvantages:

Increased complexity
Higher token usage
Increased latency

Reflection and RAG Systems

Reflection is extremely valuable in RAG applications.

The model can evaluate:

Whether retrieved documents are relevant
Whether grounding data supports conclusions
Whether citations are accurate
Whether the answer contains unsupported claims

This reduces hallucinations.

Grounding Validation

A reflection stage may ask:

Did the answer use retrieved documents?
Are citations valid?
Is every factual statement supported?
Was information invented?

This helps enterprise AI systems maintain trust.

Prompt Engineering for Reflection

Effective reflection depends heavily on prompt design.

Examples:

Reflection Prompt

“Review the answer and identify any logical inconsistencies, unsupported assumptions, or missing details.”

Hallucination Detection Prompt

“Determine whether any statements are unsupported by the provided documents.”

Safety Evaluation Prompt

“Check whether the response violates safety or compliance policies.”

Chain-of-Thought Prompting Strategies

Zero-Shot CoT

Prompt:

“Think step-by-step.”

Simple but effective.

Few-Shot CoT

Provide examples of step-by-step reasoning before asking the model to solve a problem.

Advantages:

Higher consistency
Better reasoning quality
Improved task adaptation

Structured Reasoning Prompts

Prompts explicitly require sections such as:

Problem analysis
Assumptions
Step-by-step reasoning
Final conclusion

This improves traceability.

Hidden vs Visible Chain-of-Thought

Visible Chain-of-Thought

The reasoning is shown to the user.

Advantages:

Transparency
Easier debugging
Better educational experiences

Disadvantages:

Longer outputs
Potential exposure of internal reasoning

Hidden Chain-of-Thought

The model reasons internally but only returns the final answer.

Advantages:

Cleaner user experience
Better security
Reduced information leakage

Many production systems prefer hidden reasoning.

Reflection and Safety

Reflection systems can improve AI safety.

The model can:

Detect unsafe instructions
Identify policy violations
Refuse harmful actions
Validate outputs before execution
Detect prompt injection attempts

This is critical for autonomous agents.

Approval Loops

Some workflows combine reflection with human approval.

Examples:

Financial transactions
Infrastructure changes
Healthcare recommendations
Security operations
Legal document generation

Flow:

Agent proposes action
Reflection validates action
Human approves action
Execution occurs

This creates safer semiautonomous systems.

Reflection for Code Generation

Reflection significantly improves AI-generated code.

The model can:

Detect syntax errors
Check logic
Validate APIs
Review security issues
Improve readability
Detect missing edge cases

Self-critique loops are widely used in AI coding assistants.

Error Analysis

Developers should analyze:

Reflection failures
False positives
False negatives
Incorrect critiques
Loop instability
Excessive token consumption

Error analysis helps optimize reflection pipelines.

Performance Considerations

Reflection systems improve quality but increase:

Latency
Token usage
Cost
Infrastructure complexity

Developers must balance:

Accuracy
Speed
Cost
User experience

Cost Optimization Strategies

Common optimization approaches include:

Using smaller evaluator models
Limiting reflection passes
Triggering reflection only for high-risk tasks
Using lightweight safety evaluators
Caching evaluations
Performing selective validation

Reflection Metrics

Important metrics include:

Metric	Description
Hallucination Rate	Frequency of fabricated information
Grounding Accuracy	Correct use of retrieved data
Safety Violation Rate	Unsafe outputs detected
Revision Success Rate	Improvement after critique
Tool Accuracy	Correct tool selection
Reasoning Quality	Quality of logical steps
User Satisfaction	Human feedback quality

Azure AI Foundry Evaluation Features

Azure AI Foundry supports:

Evaluation pipelines
Prompt flow orchestration
Safety evaluations
Groundedness evaluations
Relevance evaluations
Retrieval quality analysis
Monitoring dashboards
Responsible AI instrumentation

These capabilities help operationalize reflection-based AI systems.

Common Mistakes

Overusing Reflection

Too many critique loops can:

Increase latency
Increase cost
Cause output degradation
Produce repetitive answers

Weak Critique Prompts

Poor prompts lead to weak evaluations.

Prompts should clearly specify:

Evaluation criteria
Expected format
Safety requirements
Grounding expectations

Ignoring Grounding Validation

Even well-written responses may still hallucinate.

Always validate grounding in enterprise systems.

Lack of Human Oversight

High-risk systems should include human review workflows.

Best Practices

Use Reflection Selectively

Apply deeper evaluation only where needed.

Separate Generation and Evaluation

Use different prompts or models for evaluation.

Ground Responses with Trusted Data

Combine reflection with RAG architectures.

Monitor Reflection Performance

Track:

Accuracy
Safety
Cost
Latency
Evaluation quality

Use Safety Filters Together with Reflection

Reflection complements but does not replace:

Content moderation
Safety classifiers
Governance controls
Access restrictions

AI-103 Exam Tips

For the AI-103 exam, focus heavily on:

Reflection workflows
Chain-of-thought reasoning
Self-critique loops
Grounding validation
Hallucination reduction
Agent evaluation strategies
Azure AI Foundry orchestration
Prompt engineering for reasoning
Evaluation pipelines
Safety-aware AI architectures

You should understand:

When to use reflection
Tradeoffs between quality and cost
How reflection improves agents
How CoT improves reasoning
How evaluators validate outputs
How grounding checks reduce hallucinations

Summary

Model reflection, chain-of-thought evaluations, and self-critique loops are foundational techniques for building reliable generative AI systems.

These approaches improve:

Accuracy
Safety
Grounding quality
Reasoning transparency
Agent reliability
Workflow correctness

Azure AI Foundry enables developers to operationalize these techniques through:

Prompt flows
Evaluators
Monitoring systems
Safety pipelines
Agent orchestration
Retrieval systems
Responsible AI tooling

For the AI-103 exam, candidates should understand both the conceptual foundations and practical implementation patterns for reflection-driven AI systems.

Practice Exam Questions

Question 1

What is the primary purpose of model reflection in generative AI systems?

A. Reduce GPU memory usage
B. Improve output quality through self-evaluation
C. Replace retrieval systems entirely
D. Eliminate all hallucinations automatically

Answer

B. Improve output quality through self-evaluation

Explanation

Model reflection enables the AI system to review and improve its own responses before returning final output.

Question 2

What is chain-of-thought prompting primarily designed to improve?

A. Network throughput
B. Data encryption
C. Step-by-step reasoning quality
D. Vector indexing speed

Answer

C. Step-by-step reasoning quality

Explanation

Chain-of-thought prompting encourages structured reasoning processes that improve complex problem-solving.

Question 3

Which workflow best represents a self-critique loop?

A. Retrieve → Store → Delete
B. Generate → Critique → Revise
C. Train → Deploy → Archive
D. Search → Embed → Compress

Answer

B. Generate → Critique → Revise

Explanation

Self-critique loops iteratively evaluate and improve generated outputs.

Question 4

Why are reflection systems especially important in AI agents?

A. Agents do not require prompts
B. Agents never hallucinate
C. Agents often make decisions and execute actions
D. Agents cannot use tools

Answer

C. Agents often make decisions and execute actions

Explanation

Reflection helps validate agent actions, reasoning, and tool usage before execution.

Question 5

Which technique helps validate whether a RAG response is supported by retrieved documents?

A. GPU autoscaling
B. Grounding evaluation
C. Data compression
D. Blob lifecycle policies

Answer

B. Grounding evaluation

Explanation

Grounding evaluations verify whether generated content is supported by retrieved context.

Question 6

What is a disadvantage of multi-pass reflection?

A. Reduced reasoning quality
B. Lower model accuracy
C. Increased token usage and latency
D. Inability to evaluate outputs

Answer

C. Increased token usage and latency

Explanation

Additional critique and revision passes increase computational cost and response time.

Question 7

Which approach uses a separate model to evaluate generated responses?

A. Prompt caching
B. External reflection
C. Embedding normalization
D. Token pruning

Answer

B. External reflection

Explanation

External reflection separates generation from evaluation by using another model or evaluator.

Question 8

What is a key benefit of hidden chain-of-thought reasoning?

A. Faster vector indexing
B. Improved security and reduced reasoning exposure
C. Elimination of prompts
D. Lower storage requirements

Answer

B. Improved security and reduced reasoning exposure

Explanation

Hidden reasoning avoids exposing internal decision-making to users.

Question 9

Which Azure AI Foundry capability helps operationalize reflection workflows?

A. Azure CDN
B. Prompt flow orchestration
C. Virtual WAN
D. Azure Batch rendering

Answer

B. Prompt flow orchestration

Explanation

Prompt flows enable orchestration of generation, evaluation, critique, and revision stages.

Question 10

What is the main goal of self-critique loops in generative AI systems?

A. Increase network bandwidth
B. Improve answer reliability and correctness
C. Replace all human oversight
D. Reduce storage costs

Answer

B. Improve answer reliability and correctness

Explanation

Self-critique loops improve response quality by enabling iterative evaluation and refinement.

Additional Study Resources

Microsoft Learn AI-103 Training
Azure AI Foundry documentation
Azure AI Search documentation
Azure OpenAI documentation
Responsible AI guidance for Azure AI services
Prompt engineering guidance from Microsoft Learn

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Artificial Intelligence (AI), Azure AI, Generative AI, Microsoft Certification May 25, 2026

Integrate monitoring into deployed agents, evaluate agent behavior, and perform error analysis (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build agents by using Foundry
      --> Integrate monitoring into deployed agents, evaluate agent behavior, and perform error analysis

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Monitoring, evaluation, and error analysis are critical components of production-grade AI agent systems. In the AI-103 certification exam, Microsoft expects candidates to understand how to monitor deployed agents, assess their behavior, identify failures, improve safety and reliability, and continuously optimize agent performance.

Modern AI agents are dynamic systems that can reason, retrieve information, call tools, maintain memory, and execute multistep workflows. Because of this complexity, monitoring an AI agent goes far beyond checking whether an API endpoint is online. Developers must monitor prompts, tool usage, retrieval quality, token consumption, latency, failures, safety issues, hallucinations, and overall user satisfaction.

Azure AI Foundry provides tools and integrations that help developers monitor deployed agents, evaluate outputs, perform safety evaluations, collect telemetry, and conduct root-cause analysis when problems occur.

This article covers the key AI-103 exam concepts related to:

Monitoring deployed AI agents
Agent observability
Telemetry collection
Logging and tracing
Evaluating agent behavior
Measuring quality and safety
Detecting hallucinations and grounding failures
Tool-call monitoring
Conversation analytics
Error analysis techniques
Root-cause investigation
Failure handling and resiliency
Responsible AI evaluation
Continuous improvement workflows

Why Monitoring Matters in AI Agent Systems

Traditional software systems generally behave deterministically. Given the same input, the system usually produces the same output.

AI agents behave probabilistically. Outputs may vary even when prompts are similar. Agents can also:

Use external tools
Retrieve documents
Perform reasoning steps
Maintain conversational memory
Execute actions autonomously
Interact with multiple systems

Because of this complexity, production AI systems require strong observability and monitoring capabilities.

Monitoring helps organizations:

Detect failures quickly
Identify hallucinations
Measure quality
Improve safety
Optimize costs
Detect prompt injection attempts
Analyze user satisfaction
Improve retrieval relevance
Tune prompts and workflows
Validate grounding quality
Ensure compliance and auditing

Without monitoring, developers cannot reliably improve or trust deployed AI systems.

Core Monitoring Concepts

Observability

Observability refers to the ability to understand what an AI system is doing internally based on telemetry and logs.

An observable AI system provides insight into:

Prompts
Responses
Tool calls
Retrieval results
Execution paths
Latency
Failures
Safety violations
Token usage
Model selection
User interactions

Observability enables developers to diagnose problems efficiently.

Telemetry

Telemetry is operational data collected from the AI system.

Examples include:

API response times
Number of tokens consumed
Tool invocation counts
Search query performance
Error rates
Memory usage
Agent workflow duration
Failed requests
User feedback scores

Telemetry data is often stored in:

Azure Monitor
Application Insights
Log Analytics
Event Hubs
Data Lake storage

Trace Logging

Tracing records the sequence of operations executed during an agent interaction.

A trace may include:

User prompt
System prompt
Retrieval request
Retrieved documents
Tool calls
Model response
Safety filter results
Final output

Tracing is essential for debugging multistep agent workflows.

Monitoring Deployed Agents in Azure

Azure AI Foundry Monitoring

Azure AI Foundry provides monitoring capabilities for:

Model deployments
Agent workflows
Prompt flows
Evaluation pipelines
Safety evaluations
Token usage
Latency metrics
Failure tracking

Developers can analyze:

Request success rates
Response quality
Grounding quality
Safety incidents
Performance bottlenecks

Azure Monitor

Azure Monitor collects metrics and logs across Azure resources.

Common AI monitoring scenarios include:

Monitoring API latency
Detecting spikes in failed requests
Monitoring throughput
Alerting on quota exhaustion
Monitoring infrastructure health

Azure Monitor can trigger:

Email alerts
SMS notifications
Logic Apps workflows
Incident response tickets

Application Insights

Application Insights provides detailed application telemetry.

For AI agents, it can track:

User sessions
API calls
Exceptions
Dependency failures
Custom events
Prompt execution traces
Response timing

Application Insights is commonly integrated into:

Web applications
Chatbots
Agent orchestration systems
API gateways

Log Analytics

Log Analytics enables querying and analyzing telemetry data.

Developers can:

Search logs
Build dashboards
Analyze trends
Correlate failures
Investigate incidents

Kusto Query Language (KQL) is commonly used for analysis.

Example:

			
requests
| where success == false
| summarize count() by operation_Name

Important Metrics for AI Agents

Latency

Latency measures how long it takes for the agent to respond.

High latency may be caused by:

Slow model inference
Large prompts
Slow tool APIs
Complex orchestration
Vector search delays
Network bottlenecks

Low latency is especially important for:

Customer support bots
Interactive copilots
Real-time assistants

Token Usage

Large token consumption increases cost and latency.

Developers monitor:

Prompt tokens
Completion tokens
Total tokens per session
Tokens per workflow step

Reducing token usage may involve:

Shorter prompts
Better chunking
Summarized memory
Smaller models
Context pruning

Error Rates

Error monitoring helps identify instability.

Examples:

Failed tool calls
Timeout errors
Retrieval failures
API authentication errors
Model overload conditions
Rate-limit violations

High error rates indicate reliability issues.

Throughput

Throughput measures how many requests the system can handle.

Important for:

High-scale enterprise systems
Public-facing chatbots
Large customer-service systems

User Satisfaction

User feedback is critical for evaluating agent quality.

Methods include:

Thumbs up/down feedback
Star ratings
Survey scores
Conversation abandonment rates
Escalation frequency

User feedback helps identify:

Hallucinations
Poor reasoning
Irrelevant responses
Unsafe behavior

Evaluating Agent Behavior

Why Evaluation Is Important

AI agents may appear functional while still producing:

Unsafe outputs
Incorrect reasoning
Fabricated facts
Poor tool usage
Low-quality retrieval
Biased responses

Evaluation ensures the system performs reliably.

Types of Evaluations

Quality Evaluation

Measures:

Accuracy
Completeness
Helpfulness
Relevance
Coherence

Example questions:

Did the response answer the user question?
Was the answer correct?
Was the response understandable?

Grounding Evaluation

Grounding evaluations verify whether responses are supported by retrieved data.

This is especially important in RAG systems.

Developers evaluate:

Citation accuracy
Retrieval relevance
Hallucination frequency
Source alignment

Poor grounding may indicate:

Bad chunking
Weak embeddings
Incorrect search ranking
Missing documents

Safety Evaluation

Safety evaluations identify harmful or policy-violating outputs.

Examples:

Hate speech
Violence
Self-harm content
Prompt injection success
Sensitive information leakage
Toxic responses

Azure AI safety tooling can help detect these issues.

Tool Usage Evaluation

Agents may incorrectly:

Select the wrong tool
Pass invalid parameters
Call tools too frequently
Fail to call required tools

Tool evaluation measures:

Tool selection accuracy
Parameter correctness
Tool success rates
Tool latency

Conversation Evaluation

Conversation quality evaluation measures:

Context retention
Memory quality
Conversation consistency
Turn-by-turn coherence
Goal completion success

Evaluators in Azure AI Foundry

Azure AI Foundry supports evaluators that help assess model and agent quality.

Evaluators may analyze:

Relevance
Groundedness
Coherence
Fluency
Safety
Similarity to reference answers

Evaluation pipelines may run:

During development
During testing
After deployment
Continuously in production

Detecting Hallucinations

What Is a Hallucination?

A hallucination occurs when the model generates false or fabricated information.

Examples:

Invented facts
Nonexistent citations
False calculations
Fabricated policies
Incorrect summaries

Causes of Hallucinations

Common causes include:

Weak grounding
Missing context
Poor prompts
Overly broad tasks
Outdated training data
Low retrieval quality

Hallucination Detection Techniques

Methods include:

Grounding evaluations
Citation verification
Reference-answer comparison
Human review
Fact-checking pipelines
Confidence scoring

Monitoring Retrieval Quality

In RAG systems, retrieval quality strongly affects response quality.

Developers monitor:

Search relevance
Chunk quality
Embedding effectiveness
Citation accuracy
Vector search latency
Retrieval precision
Retrieval recall

Poor retrieval causes:

Irrelevant answers
Missing context
Hallucinations
Reduced trustworthiness

Error Analysis in AI Systems

What Is Error Analysis?

Error analysis is the process of investigating failures and identifying root causes.

The goal is to improve:

Reliability
Accuracy
Safety
Performance
User experience

Common AI Agent Failure Types

Retrieval Failures

Examples:

Wrong documents retrieved
Missing relevant documents
Low-quality embeddings
Poor chunking strategy

Solutions:

Improve chunking
Use hybrid search
Tune embeddings
Improve metadata filtering

Prompt Failures

Examples:

Ambiguous prompts
Missing instructions
Weak system prompts
Excessively large prompts

Solutions:

Refine prompt templates
Add examples
Improve role instructions
Use structured outputs

Tool Invocation Failures

Examples:

Tool unavailable
Invalid parameters
Incorrect API schema
Timeout issues

Solutions:

Add retries
Validate inputs
Improve schemas
Add fallback workflows

Reasoning Failures

Examples:

Incorrect multistep logic
Incomplete planning
Contradictory outputs
Failed task sequencing

Solutions:

Break tasks into smaller steps
Use orchestration frameworks
Add verification stages
Add human approval checkpoints

Memory Failures

Examples:

Forgetting earlier conversation context
Using outdated memory
Injecting irrelevant memory

Solutions:

Summarize memory
Use memory expiration policies
Improve retrieval logic

Root-Cause Analysis

Developers use logs and traces to identify:

What failed
Where it failed
Why it failed
Which dependency caused failure

Root-cause analysis often examines:

Prompt versions
Model versions
Retrieved documents
Tool responses
System state
User inputs

A/B Testing and Continuous Improvement

A/B Testing

A/B testing compares multiple versions of:

Prompts
Models
Retrieval strategies
Tool orchestration
Agent workflows

Example:

Version A uses GPT-4
Version B uses a smaller model

Metrics are compared to determine the better approach.

Continuous Evaluation

Production AI systems should continuously evaluate:

Safety
Quality
Relevance
Cost
Latency
User satisfaction

Continuous evaluation helps detect:

Drift
Degradation
Emerging risks

Responsible AI Monitoring

Responsible AI monitoring includes:

Safety evaluations
Bias detection
Toxicity detection
Compliance auditing
Human oversight
Approval workflows

Monitoring should ensure agents:

Follow policies
Avoid harmful outputs
Respect privacy
Operate within defined constraints

Human-in-the-Loop Monitoring

High-risk systems often include human review.

Examples:

Financial recommendations
Medical suggestions
Legal analysis
Security operations

Human reviewers may:

Approve actions
Review flagged outputs
Escalate incidents
Correct model errors

Alerting and Incident Response

Monitoring systems should generate alerts for:

Increased hallucinations
Safety violations
Tool failures
Excessive latency
Rising error rates
Unusual traffic spikes

Alerts support rapid incident response.

Dashboards and Visualization

Dashboards help teams monitor AI systems visually.

Typical dashboard metrics include:

Request volume
Token consumption
Failure rates
Latency
Safety incidents
Tool usage
Retrieval quality
User ratings

Azure dashboards commonly use:

Azure Monitor
Power BI
Application Insights workbooks

Best Practices for Monitoring AI Agents

Enable Full Tracing

Capture:

Inputs
Outputs
Tool calls
Retrieval results
Safety decisions

Log Prompt Versions

Always track:

Prompt templates
System messages
Model versions

This simplifies debugging.

Evaluate Continuously

Do not evaluate only during development.

Production evaluation is essential.

Use Human Review for High-Risk Tasks

High-impact decisions should include human oversight.

Monitor Cost and Performance

Track:

Token usage
Latency
Throughput
Scaling costs

Test Failure Scenarios

Simulate:

Tool outages
Bad retrieval
Prompt injection
Rate limits
Safety attacks

AI-103 Exam Tips

For the AI-103 exam, remember these important points:

Monitoring AI agents requires more than infrastructure monitoring.
Observability includes prompts, tool calls, retrieval, memory, and outputs.
Application Insights and Azure Monitor are commonly used for telemetry.
Grounding evaluations help detect hallucinations.
Safety evaluations identify harmful outputs.
Trace logging is essential for debugging multistep workflows.
Tool-call monitoring helps identify orchestration failures.
Retrieval quality directly affects RAG system quality.
Error analysis focuses on root causes and corrective actions.
Human oversight is important in high-risk systems.

Practice Exam Questions

Question 1

What is the primary purpose of observability in AI agent systems?

A. Reduce cloud storage usage
B. Understand internal agent behavior through telemetry and logs
C. Eliminate all hallucinations
D. Increase GPU memory

Correct Answer

B. Understand internal agent behavior through telemetry and logs

Explanation

Observability helps developers understand prompts, tool calls, retrieval steps, failures, and outputs within AI systems.

Question 2

Which Azure service is commonly used for collecting application telemetry and exceptions?

A. Azure DNS
B. Azure Kubernetes Service
C. Application Insights
D. Azure Files

Correct Answer

C. Application Insights

Explanation

Application Insights collects telemetry, traces, exceptions, performance metrics, and dependency information.

Question 3

What is a hallucination in generative AI?

A. A successful retrieval operation
B. A fabricated or incorrect model output
C. A network timeout
D. A token optimization method

Correct Answer

B. A fabricated or incorrect model output

Explanation

Hallucinations occur when a model generates false or unsupported information.

Question 4

Which evaluation type verifies whether model responses are supported by retrieved documents?

A. Infrastructure evaluation
B. Throughput evaluation
C. Grounding evaluation
D. Scaling evaluation

Correct Answer

C. Grounding evaluation

Explanation

Grounding evaluations assess whether responses align with retrieved sources.

Question 5

Which issue is most likely caused by poor retrieval quality in a RAG system?

A. GPU overheating
B. Irrelevant or incomplete answers
C. Faster response times
D. Lower token usage

Correct Answer

B. Irrelevant or incomplete answers

Explanation

Poor retrieval quality reduces the relevance and accuracy of generated answers.

Question 6

What is the purpose of trace logging in AI workflows?

A. Increase storage costs
B. Encrypt prompts
C. Record workflow execution details for debugging
D. Replace vector search

Correct Answer

C. Record workflow execution details for debugging

Explanation

Trace logging captures execution steps, tool calls, retrieval results, and model outputs.

Question 7

Which metric directly measures how quickly an AI agent responds?

A. Recall
B. Latency
C. Groundedness
D. Fluency

Correct Answer

B. Latency

Explanation

Latency measures response time.

Question 8

What is a common strategy for improving reliability in high-risk AI systems?

A. Removing all monitoring
B. Disabling safety filters
C. Adding human-in-the-loop approvals
D. Eliminating trace logs

Correct Answer

C. Adding human-in-the-loop approvals

Explanation

Human review improves oversight and reduces risks in sensitive workflows.

Question 9

Which type of failure occurs when an agent selects the wrong API or tool?

A. Memory failure
B. Retrieval failure
C. Tool invocation failure
D. Scaling failure

Correct Answer

C. Tool invocation failure

Explanation

Incorrect tool selection or invalid tool parameters are tool invocation failures.

Question 10

Why is continuous evaluation important in production AI systems?

A. To permanently lock model behavior
B. To detect degradation, drift, and emerging risks
C. To reduce all network traffic
D. To eliminate telemetry collection

Correct Answer

B. To detect degradation, drift, and emerging risks

Explanation

Continuous evaluation helps organizations identify quality degradation, safety issues, and changing system behavior over time.

Final Thoughts

Monitoring and evaluating AI agents is one of the most important responsibilities for AI developers working with Azure AI Foundry. Production AI systems require continuous observability, telemetry analysis, safety evaluation, grounding validation, and error analysis.

For the AI-103 exam, candidates should understand:

How to monitor AI agents
Which Azure services support observability
How to evaluate AI quality and safety
How to detect hallucinations
How to analyze failures
How to improve agent reliability and performance

Strong monitoring and evaluation practices are essential for building trustworthy, scalable, and production-ready AI systems.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Azure AI, Microsoft Certification May 25, 2026

Implement orchestrated multi-agent solutions (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build agents by using Foundry
      --> Implement orchestrated multi-agent solutions

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

As AI systems become more advanced, organizations increasingly use multiple AI agents working together rather than relying on a single monolithic model.

Multi-agent systems allow specialized agents to:

Collaborate
Delegate tasks
Share information
Coordinate workflows
Solve complex business problems

Azure AI Foundry provides orchestration capabilities that enable developers to design and implement coordinated multi-agent architectures.

For the AI-103: Develop AI Apps and Agents on Azure certification exam, understanding orchestrated multi-agent solutions is an important skill area.

What Is a Multi-Agent System?

A multi-agent system consists of:

Multiple AI agents
Coordinated workflows
Shared objectives
Task delegation mechanisms
Communication pathways

Each agent typically performs a specialized role.

Why Use Multi-Agent Architectures?

Multi-agent systems improve:

Scalability
Modularity
Specialization
Reliability
Workflow efficiency

Single-Agent vs Multi-Agent Systems

Single-Agent Systems

Single-agent systems:

Handle all responsibilities centrally
Use one model for all tasks
Are simpler to implement

However, they may struggle with:

Complex workflows
Large-scale orchestration
Specialized reasoning

Multi-Agent Systems

Multi-agent systems:

Separate responsibilities
Assign specialized tasks
Coordinate multiple workflows
Improve maintainability

Common Multi-Agent Roles

Examples of specialized agents include:

Research agents
Retrieval agents
Planning agents
Coding agents
Compliance agents
Validation agents
Summarization agents
Customer support agents

Agent Specialization

Specialized agents often outperform general-purpose agents because:

Prompts can be optimized
Tools can be restricted
Workflows become more focused
Context becomes more manageable

Orchestration

Orchestration coordinates:

Agent communication
Task delegation
Workflow sequencing
State management
Tool usage

What Is an Orchestrator?

An orchestrator is a coordinating component that:

Routes tasks
Selects agents
Manages workflows
Tracks execution state
Aggregates outputs

Centralized Orchestration

In centralized orchestration:

One orchestrator controls workflows
Agents report to a central controller
Execution is easier to monitor

Decentralized Orchestration

In decentralized orchestration:

Agents communicate directly
Coordination is distributed
Systems may scale more dynamically

Hierarchical Agent Systems

Hierarchical systems use:

Supervisor agents
Worker agents
Nested workflows

The supervisor assigns and validates tasks.

Agent Communication

Agents communicate by:

Passing messages
Sharing outputs
Updating workflow state
Exchanging structured data

Shared Context

Multi-agent systems may share:

Conversation history
Retrieved documents
Task state
Memory stores
Workflow variables

Conversation State Management

State management tracks:

Current workflow stage
Completed actions
Pending tasks
Agent outputs

Workflow Coordination

Workflow coordination defines:

Execution order
Conditional branching
Retry behavior
Escalation logic

Sequential Workflows

Sequential workflows execute agents in order.

Example:

Retrieval agent
Validation agent
Summarization agent
Approval agent

Parallel Workflows

Parallel workflows allow multiple agents to:

Execute simultaneously
Process independent tasks
Improve performance

Conditional Workflows

Conditional workflows branch based on:

User input
Confidence scores
Validation results
Business rules

Dynamic Routing

Dynamic routing enables orchestrators to:

Select agents at runtime
Adapt workflows dynamically
Optimize execution paths

Planning Agents

Planning agents:

Break tasks into subtasks
Determine execution order
Coordinate tool usage
Guide workflow progression

Task Delegation

Task delegation assigns work to specialized agents.

Examples:

Retrieval tasks
Compliance validation
Data analysis
Report generation

Tool-Augmented Multi-Agent Systems

Agents may use tools such as:

APIs
Search systems
Databases
Workflow engines
Custom functions

Retrieval Agents

Retrieval agents specialize in:

Searching enterprise data
Retrieving documents
Querying vector stores
Performing semantic search

Validation Agents

Validation agents may:

Detect hallucinations
Verify citations
Enforce compliance
Apply safety checks

Compliance Agents

Compliance agents help enforce:

Regulatory requirements
Security policies
Governance standards
Responsible AI rules

Human-in-the-Loop Systems

Some workflows require:

Human approval
Escalation review
Manual validation

before execution continues.

Memory in Multi-Agent Systems

Agents may use:

Short-term memory
Long-term memory
Shared memory
Retrieval-based memory

Shared Memory Systems

Shared memory allows agents to:

Access common information
Coordinate tasks
Maintain consistency

Long-Term Memory

Long-term memory stores:

Historical interactions
User preferences
Prior workflow results
Persistent context

Vector Memory

Vector memory uses embeddings to:

Store semantic information
Retrieve relevant history
Improve contextual continuity

Retrieval-Augmented Multi-Agent Systems

Multi-agent systems often integrate:

Azure AI Search
Vector search
Semantic retrieval
Grounding pipelines

Azure AI Search in Multi-Agent Systems

Azure AI Search supports:

Hybrid search
Semantic ranking
Vector indexing
Enterprise retrieval

Grounded Agent Responses

Grounded systems use retrieved evidence to:

Improve factual accuracy
Reduce hallucinations
Increase trustworthiness

Multi-Agent Reasoning

Complex reasoning may involve:

Planning agents
Research agents
Verification agents
Synthesis agents

working together.

Example Multi-Agent Workflow

Enterprise Research Assistant

Workflow:

Planner agent analyzes user request
Retrieval agent searches enterprise documents
Research agent summarizes findings
Validation agent checks citations
Compliance agent reviews policy concerns
Final response agent generates answer

Multi-Agent Coordination Challenges

Challenges include:

State synchronization
Latency
Tool conflicts
Redundant work
Workflow complexity

Latency Management

Latency can increase because:

Multiple agents execute sequentially
Retrieval systems add overhead
APIs require network calls

Optimization Strategies

Optimization techniques include:

Parallel execution
Response caching
Efficient retrieval
Selective tool invocation
Lightweight models for subtasks

Small Models in Multi-Agent Systems

Smaller models may handle:

Classification
Routing
Validation
Tool selection

while larger models perform complex reasoning.

Cost Optimization

Organizations may reduce costs by:

Using specialized lightweight agents
Limiting unnecessary tool calls
Reducing prompt size
Caching retrieval results

Monitoring Multi-Agent Systems

Monitoring should include:

Agent performance
Workflow success rates
Latency
Tool failures
Retrieval quality
Safety events

Logging and Traceability

Logs should capture:

Agent decisions
Tool invocations
Retrieval outputs
Workflow paths
Human approvals

Observability

Observability enables teams to:

Diagnose failures
Analyze workflows
Improve orchestration
Monitor reasoning quality

Security Considerations

Multi-agent systems require:

Authentication
Authorization
Role-based access control (RBAC)
Managed identities
Secure tool access

Least Privilege Access

Each agent should receive:

Only required permissions
Restricted tool access
Scoped credentials

Responsible AI Considerations

Organizations should implement:

Safety filters
Approval workflows
Oversight controls
Audit logging
Content moderation

Failure Recovery

Recovery mechanisms may include:

Retries
Escalation paths
Fallback agents
Human intervention

Agent Evaluation

Organizations should evaluate:

Task completion accuracy
Hallucination rates
Retrieval quality
Workflow reliability
Safety compliance

Azure AI Foundry and Multi-Agent Solutions

Azure AI Foundry supports:

Agent development
Tool integration
Workflow orchestration
Model deployment
Retrieval integration
Monitoring and evaluation

Common AI-103 Exam Tips

Understand Agent Roles

Know how specialized agents:

Coordinate
Delegate tasks
Use tools
Share context

Understand Orchestration Patterns

Know:

Sequential workflows
Parallel workflows
Hierarchical systems
Dynamic routing

Learn Retrieval Integration

Understand:

Azure AI Search
RAG
Vector search
Embeddings
Grounding

Learn Monitoring Concepts

Understand:

Trace logging
Workflow monitoring
Observability
Safety monitoring

Summary

Orchestrated multi-agent systems enable:

Specialized AI workflows
Coordinated reasoning
Tool integration
Enterprise-scale automation

For the AI-103 exam, you should understand:

Multi-agent architectures
Agent orchestration
Workflow coordination
Task delegation
Shared memory
Retrieval integration
Planning agents
Validation agents
Compliance workflows
Dynamic routing
Monitoring and observability
Responsible AI controls

These concepts are foundational for enterprise AI agent development in Azure AI Foundry.

Practice Exam Questions

Question 1

What is a primary advantage of multi-agent systems?

A. Elimination of workflows
B. Agent specialization and task coordination
C. Removal of retrieval systems
D. Elimination of APIs

Answer

B. Agent specialization and task coordination

Explanation

Multi-agent systems improve modularity and specialization.

Question 2

What is the role of an orchestrator in a multi-agent system?

A. Replace all agents
B. Coordinate workflows and manage execution
C. Disable APIs
D. Eliminate memory usage

Answer

B. Coordinate workflows and manage execution

Explanation

Orchestrators route tasks and coordinate agent interactions.

Question 3

Which workflow type allows multiple agents to execute simultaneously?

A. Sequential workflow
B. Parallel workflow
C. Static workflow
D. Manual workflow

Answer

B. Parallel workflow

Explanation

Parallel workflows improve performance by enabling concurrent execution.

Question 4

What is a common role for a retrieval agent?

A. GPU maintenance
B. Searching enterprise knowledge sources
C. Managing DNS records
D. Updating operating systems

Answer

B. Searching enterprise knowledge sources

Explanation

Retrieval agents specialize in search and document retrieval.

Question 5

Why are validation agents useful?

A. They eliminate monitoring
B. They verify outputs and reduce hallucinations
C. They remove orchestration logic
D. They disable APIs

Answer

B. They verify outputs and reduce hallucinations

Explanation

Validation agents improve reliability and compliance.

Question 6

What is shared memory in a multi-agent system?

A. A GPU cache
B. A common context accessible by multiple agents
C. A networking appliance
D. A firewall rule set

Answer

B. A common context accessible by multiple agents

Explanation

Shared memory improves coordination between agents.

Question 7

Which Azure service is commonly used for enterprise retrieval in multi-agent systems?

A. Azure AI Search
B. Azure Backup
C. Azure Monitor Agent
D. Azure VPN Gateway

Answer

A. Azure AI Search

Explanation

Azure AI Search supports semantic, vector, and hybrid retrieval.

Question 8

What is dynamic routing?

A. Static API configuration
B. Selecting agents at runtime based on workflow needs
C. Replacing retrieval systems
D. Eliminating orchestrators

Answer

B. Selecting agents at runtime based on workflow needs

Explanation

Dynamic routing enables adaptive workflows.

Question 9

Why might organizations use small models in multi-agent systems?

A. To increase hallucinations
B. To reduce cost and handle lightweight subtasks
C. To eliminate orchestration
D. To disable memory

Answer

B. To reduce cost and handle lightweight subtasks

Explanation

Small models are efficient for routing and classification tasks.

Question 10

What should organizations monitor in multi-agent solutions?

A. Only GPU temperatures
B. Workflow reliability, retrieval quality, latency, and safety events
C. Only token counts
D. Only firewall rules

Answer

B. Workflow reliability, retrieval quality, latency, and safety events

Explanation

Monitoring ensures reliable and safe multi-agent operations.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Artificial Intelligence (AI), Azure AI, Generative AI, Microsoft Certification May 25, 2026

Integrate agent tools, including APIs, knowledge stores, search, Content Understanding, and custom functions (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build agents by using Foundry
      --> Integrate agent tools, including APIs, knowledge stores, search, Content Understanding, and custom functions

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI agents are capable of far more than generating text.

Enterprise AI agents can:

Access business systems
Retrieve enterprise knowledge
Search documents
Understand multimodal content
Execute workflows
Interact with APIs
Use custom functions

These capabilities are possible because modern agentic systems integrate external tools.

Azure AI Foundry provides orchestration and integration capabilities for building tool-augmented AI agents.

For the AI-103: Develop AI Apps and Agents on Azure certification exam, understanding how agents integrate with:

APIs
Knowledge stores
Search systems
Content understanding services
Custom functions

is a major exam objective.

What Are Agent Tools?

Agent tools are external capabilities that agents can invoke to:

Retrieve information
Perform actions
Execute workflows
Interact with systems

Why Tool Integration Matters

LLMs alone cannot:

Access real-time business data
Execute transactions
Query live systems
Retrieve private enterprise information

Tool integration enables these capabilities.

Types of Agent Tools

Common agent tools include:

APIs
Databases
Search services
Vector stores
Content understanding systems
Workflow engines
Custom functions
External applications

Tool-Augmented Agents

Tool-augmented agents combine:

Language reasoning
Retrieval systems
External actions
Workflow orchestration

APIs in Agent Systems

APIs are among the most common tools used by AI agents.

APIs allow agents to:

Retrieve data
Update systems
Trigger workflows
Access cloud services

Common API Integration Scenarios

Examples include:

CRM systems
ERP systems
Ticketing systems
Email services
Calendar systems
Inventory systems
Financial platforms

REST APIs

Many agent integrations use REST APIs.

REST APIs commonly support:

GET operations
POST operations
PUT operations
DELETE operations

API Authentication

Agent systems may authenticate using:

API keys
OAuth tokens
Managed identities
Microsoft Entra ID

Managed Identity Integration

Managed identities allow applications to:

Authenticate securely
Avoid storing secrets
Access Azure resources safely

Function-Calling

Function-calling allows models to:

Invoke tools dynamically
Generate structured requests
Execute external operations

Tool Schemas

Tool schemas define:

Tool names
Input parameters
Data types
Required fields
Expected outputs

Structured Tool Invocation

Structured invocation improves:

Reliability
Validation
Automation
Predictability

Knowledge Stores

Knowledge stores provide persistent enterprise information for retrieval.

Knowledge stores may contain:

Documents
Policies
Product manuals
Research data
Historical records

Why Knowledge Stores Matter

Knowledge stores allow agents to:

Access enterprise-specific information
Ground responses
Improve factual accuracy

Knowledge Sources

Agents may connect to:

Azure AI Search
SharePoint
SQL databases
Blob storage
Cosmos DB
Data Lake storage
Vector databases

Retrieval-Augmented Generation (RAG)

RAG combines:

Retrieval systems
Generative models

Retrieved data is added to prompts to improve grounded responses.

Search Systems in Agent Architectures

Search systems allow agents to:

Retrieve relevant content
Find documents
Search enterprise knowledge
Improve response quality

Azure AI Search

Azure AI Search is commonly used for:

Keyword search
Vector search
Hybrid search
Semantic ranking

Semantic Search

Semantic search focuses on:

Meaning
Context
Intent

rather than exact keyword matches.

Vector Search

Vector search uses embeddings to:

Identify semantic similarity
Retrieve related content
Improve retrieval quality

Hybrid Search

Hybrid search combines:

Keyword search
Vector search

This improves search relevance.

Embeddings

Embeddings are vector representations of data.

Embeddings support:

Semantic retrieval
Similarity comparison
Vector indexing

Retrieval Pipelines

Retrieval pipelines commonly include:

Data ingestion
Chunking
Embedding generation
Indexing
Retrieval
Reranking

Grounded Responses

Grounded responses are generated using retrieved evidence.

Grounding improves:

Accuracy
Explainability
Trustworthiness

Content Understanding

Content understanding systems allow agents to analyze:

Images
Documents
Audio
Video
Forms
Structured and unstructured content

Multimodal Processing

Multimodal systems process multiple content types simultaneously.

Examples include:

Text + images
Text + audio
Documents + tables

Azure AI Content Understanding Capabilities

Agents may integrate with services for:

OCR
Image analysis
Speech recognition
Document intelligence
Form extraction
Video analysis

OCR Integration

Optical Character Recognition (OCR) extracts text from:

Images
PDFs
Scanned documents

Document Intelligence

Document intelligence systems can extract:

Key-value pairs
Tables
Forms
Structured business data

Image Understanding

Agents may analyze images for:

Object detection
Caption generation
Classification
Scene understanding

Speech Integration

Speech systems enable:

Speech-to-text
Text-to-speech
Voice assistants
Audio analysis

Custom Functions

Custom functions extend agent capabilities beyond built-in tools.

Custom functions may:

Execute business logic
Integrate proprietary systems
Trigger workflows
Process specialized data

Examples of Custom Functions

Examples include:

Risk scoring
Inventory forecasting
Pricing calculations
Compliance validation
Workflow automation

Designing Custom Functions

Good custom functions should:

Be narrowly scoped
Use structured parameters
Return predictable outputs
Support validation

Error Handling for Tools

Agent systems should handle:

API failures
Timeouts
Invalid responses
Authentication errors
Missing data

Retry Logic

Retry mechanisms improve resilience when:

APIs temporarily fail
Services throttle requests
Network issues occur

Tool Selection Logic

Agents may decide:

Whether a tool is needed
Which tool to invoke
When to retrieve information
How to sequence actions

Multi-Tool Orchestration

Advanced agents may coordinate:

Search systems
APIs
Memory systems
Custom functions
Workflow engines

Workflow Coordination

Agent workflows may include:

Retrieve enterprise data
Analyze content
Call APIs
Generate summaries
Execute actions

Conversation Memory Integration

Agents may combine tools with:

Short-term memory
Long-term memory
Context tracking
Session persistence

Security Considerations

Secure tool integration requires:

Authentication
Authorization
RBAC
Managed identities
Secret management
Network controls

Least Privilege Principle

Agents should receive:

Minimal required permissions
Restricted tool access
Scoped credentials

Monitoring Tool Usage

Organizations should monitor:

Tool invocation frequency
API failures
Unauthorized actions
Retrieval quality
Workflow success rates

Logging and Auditing

Logs may capture:

Tool calls
API requests
Workflow execution
Retrieved sources
User interactions

Responsible AI Considerations

Organizations should implement:

Safety filters
Guardrails
Human oversight
Approval workflows
Content moderation

Human-in-the-Loop Workflows

Sensitive operations may require:

Human review
Approval checkpoints
Escalation processes

Performance Optimization

Optimization strategies include:

Caching
Query optimization
Efficient chunking
Parallel tool execution
Response streaming

Real-World Scenario

Scenario: Enterprise Legal Assistant

Requirements:

Search legal documents
Retrieve contract clauses
Analyze uploaded PDFs
Query compliance systems
Generate summaries

Recommended Design:

Azure AI Search for retrieval
OCR and document intelligence
Function-calling for compliance APIs
Conversation memory for continuity
Approval workflows for legal actions

Common AI-103 Exam Tips

Understand Tool Integration

Know:

APIs
Function-calling
Tool schemas
Tool orchestration

Learn Retrieval Concepts

Understand:

RAG
Vector search
Embeddings
Hybrid search
Grounding

Understand Content Understanding

Know:

OCR
Document intelligence
Image analysis
Speech services
Multimodal processing

Learn Security Concepts

Understand:

Managed identities
RBAC
Least privilege
Authentication methods

Summary

Modern AI agents integrate:

APIs
Search systems
Knowledge stores
Content understanding services
Custom functions
Workflow orchestration

For the AI-103 exam, you should understand:

Tool integration
Function-calling
Tool schemas
Retrieval systems
Azure AI Search
Embeddings
Grounding
OCR and document intelligence
Multimodal processing
Custom business functions
Workflow orchestration
Monitoring and governance

These capabilities are foundational for enterprise AI agent systems built with Azure AI Foundry.

Practice Exam Questions

Question 1

Why do AI agents integrate external tools?

A. To eliminate workflows
B. To access live systems and execute actions
C. To remove retrieval systems
D. To disable APIs

Answer

B. To access live systems and execute actions

Explanation

External tools allow agents to retrieve data and perform operations.

Question 2

What is the purpose of function-calling?

A. Replace search systems
B. Allow models to invoke external tools dynamically
C. Remove authentication requirements
D. Eliminate embeddings

Answer

B. Allow models to invoke external tools dynamically

Explanation

Function-calling enables structured interaction with external systems.

Question 3

What information is typically defined in a tool schema?

A. GPU temperatures
B. Input parameters and expected outputs
C. Firewall rules only
D. VM configurations only

Answer

B. Input parameters and expected outputs

Explanation

Tool schemas standardize tool interactions.

Question 4

Which Azure service is commonly used for vector and hybrid search?

A. Azure Virtual WAN
B. Azure AI Search
C. Azure Batch
D. Azure Policy

Answer

B. Azure AI Search

Explanation

Azure AI Search supports semantic, vector, and hybrid search.

Question 5

What is the purpose of embeddings?

A. Replace APIs entirely
B. Represent data semantically for similarity comparison
C. Eliminate vector indexes
D. Remove retrieval systems

Answer

B. Represent data semantically for similarity comparison

Explanation

Embeddings support semantic retrieval.

Question 6

What is a key benefit of grounded responses?

A. Reduced monitoring needs
B. Improved factual accuracy and trustworthiness
C. Elimination of search systems
D. Removal of citations

Answer

B. Improved factual accuracy and trustworthiness

Explanation

Grounded systems use retrieved evidence to improve reliability.

Question 7

Which capability extracts text from scanned documents?

A. Vector indexing
B. OCR
C. Hybrid search
D. Tokenization

Answer

B. OCR

Explanation

OCR extracts text from images and scanned files.

Question 8

Why are managed identities important in agent systems?

A. They increase hallucinations
B. They allow secure authentication without stored secrets
C. They eliminate RBAC
D. They disable APIs

Answer

B. They allow secure authentication without stored secrets

Explanation

Managed identities improve security and credential management.

Question 9

What is an example of a custom function?

A. A GPU driver update
B. A proprietary pricing calculation workflow
C. A firewall appliance
D. A VM snapshot

Answer

B. A proprietary pricing calculation workflow

Explanation

Custom functions implement specialized business logic.

Question 10

What should organizations monitor in tool-augmented agents?

A. Only CPU temperatures
B. Tool usage, API failures, retrieval quality, and workflow success
C. Only vector dimensions
D. Only prompt length

Answer

B. Tool usage, API failures, retrieval quality, and workflow success

Explanation

Monitoring improves reliability, governance, and operational visibility.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Azure AI, Microsoft Certification May 25, 2026

Build agents that integrate retrieval, function-calling, and conversation memory (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build agents by using Foundry
      --> Build agents that integrate retrieval, function-calling, and conversation memory

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI agents are far more capable than traditional chatbots.

Today’s enterprise AI agents can:

Retrieve enterprise knowledge
Call APIs and tools
Maintain memory across conversations
Perform multistep workflows
Coordinate reasoning and actions

Azure AI Foundry provides the infrastructure and orchestration capabilities needed to build these advanced agentic systems.

For the AI-103: Develop AI Apps and Agents on Azure certification exam, understanding how to build agents that integrate:

Retrieval
Function-calling
Conversation memory

is extremely important.

These capabilities are foundational to enterprise generative AI systems.

What Is an AI Agent?

An AI agent is an AI-powered system capable of:

Understanding goals
Maintaining context
Using tools
Retrieving information
Performing actions
Adapting to new inputs

Agents extend beyond simple prompt-response interactions.

Core Components of Modern Agents

Modern agents commonly include:

Large language models (LLMs)
Retrieval systems
Tool integrations
Function-calling frameworks
Memory systems
Workflow orchestration
Safety controls

Retrieval in Agent Systems

Retrieval allows agents to:

Access external knowledge
Ground responses in enterprise data
Improve factual accuracy
Reduce hallucinations

Why Retrieval Matters

LLMs are trained on static datasets.

Without retrieval:

Models may lack current information
Enterprise-specific knowledge may be unavailable
Hallucinations become more likely

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) combines:

Search and retrieval systems
LLM reasoning and generation

RAG allows agents to generate responses using retrieved content.

Typical RAG Workflow

A common RAG workflow includes:

User submits a query
Query is converted to embeddings
Search retrieves relevant documents
Documents are added to prompts
LLM generates grounded responses

Knowledge Sources for Retrieval

Agents may retrieve data from:

Azure AI Search
Vector databases
SQL databases
Document repositories
SharePoint
Blob storage
Knowledge bases

Vector Search

Vector search enables semantic retrieval.

Instead of keyword matching only, vector search finds:

Meaning
Similarity
Contextual relationships

Embeddings

Embeddings are numerical vector representations of text or data.

Embeddings help systems:

Measure semantic similarity
Perform vector search
Improve retrieval relevance

Chunking Strategies

Documents are often split into smaller chunks before indexing.

Chunking improves:

Retrieval precision
Context quality
Token efficiency

Retrieval Pipelines

Retrieval pipelines commonly include:

Data ingestion
Chunking
Embedding generation
Indexing
Query retrieval
Reranking

Hybrid Search

Hybrid search combines:

Keyword search
Vector search

This improves search quality.

Grounding Responses

Grounding means generating responses using retrieved evidence.

Grounded systems are:

More accurate
More explainable
More reliable

Citation and Source Attribution

Agents may include:

Source links
Document citations
Retrieved evidence

This improves transparency.

Function-Calling in Agent Systems

Function-calling allows models to invoke:

APIs
Services
Workflows
Databases
External tools

Why Function-Calling Matters

LLMs alone cannot:

Access live systems
Execute actions
Retrieve dynamic business data

Function-calling bridges this gap.

Examples of Functions

Common functions include:

Get weather data
Retrieve customer records
Create support tickets
Query inventory systems
Send emails
Schedule meetings

Tool Schemas

Function-calling relies on structured tool schemas.

Schemas define:

Tool names
Parameters
Data types
Required fields
Expected outputs

Example Function Schema

Example:

Function: GetOrderStatus

Inputs:

OrderID
CustomerID

Outputs:

Shipping status
Estimated delivery date

Structured Tool Invocation

Structured tool invocation improves:

Reliability
Validation
Automation
Error handling

Function Selection Logic

Agents may decide:

Whether tools are needed
Which tools to invoke
When to call functions
How to sequence operations

Multi-Tool Workflows

Advanced agents may orchestrate:

Multiple tools
Sequential workflows
Conditional logic
Parallel execution

Example Multi-Tool Workflow

Example:

Retrieve customer data
Query billing system
Generate summary
Create support ticket
Send notification

Tool Safety Controls

Organizations should control:

Which tools agents can access
Which users may trigger actions
Which workflows require approval

Human-in-the-Loop Approvals

High-risk operations may require:

Human review
Approval checkpoints
Escalation workflows

Conversation Memory

Conversation memory allows agents to:

Maintain context
Track interactions
Remember prior information
Continue workflows

Why Memory Matters

Without memory:

Conversations become disconnected
Users repeat information
Workflow continuity breaks

Types of Memory

Common memory types include:

Short-term memory
Long-term memory
Episodic memory
Semantic memory

Short-Term Memory

Short-term memory stores:

Recent prompts
Recent responses
Current task state

Long-Term Memory

Long-term memory stores:

User preferences
Historical interactions
Persistent context

Stateful vs Stateless Agents

Stateless Agents

Do not retain memory between sessions.

Benefits:

Simpler architecture
Lower storage requirements

Stateful Agents

Maintain context and conversation history.

Benefits:

Better user experiences
Improved multistep reasoning

Context Window Limitations

LLMs have limited context windows.

Applications must manage:

Token usage
Conversation length
Historical context

Memory Management Strategies

Common strategies include:

Rolling conversation windows
Summarized history
Vector memory retrieval
Persistent storage systems

Vector Memory

Conversation history may be stored as embeddings.

This enables:

Semantic memory retrieval
Long-term contextual recall
Personalized interactions

Retrieval-Based Memory

Agents may retrieve:

Prior conversations
Historical workflow data
Previous decisions

Persistent Memory Storage

Persistent memory may use:

Databases
Search indexes
Vector stores
Cloud storage

Agent Orchestration

Orchestration coordinates:

Retrieval systems
Function-calling
Memory systems
Workflow execution

Agent Reasoning Loops

Agents may perform iterative reasoning:

Analyze request
Retrieve information
Call tools
Evaluate outputs
Continue reasoning
Generate response

Workflow State Management

Agents may track:

Active tasks
Tool outputs
Pending actions
Workflow progress

Azure AI Foundry and Agent Development

Azure AI Foundry supports:

Model deployment
Retrieval integration
Agent orchestration
Prompt flows
Evaluation pipelines
Monitoring and governance

Azure AI Search in Agent Systems

Azure AI Search commonly provides:

Vector indexing
Semantic ranking
Hybrid search
Enterprise retrieval

Prompt Engineering for Agents

Effective prompts define:

Agent role
Behavioral expectations
Tool usage rules
Safety constraints

Grounded Prompt Construction

Grounded prompts may include:

Retrieved documents
Citations
Tool outputs
Prior conversation context

Monitoring Agent Systems

Organizations should monitor:

Retrieval relevance
Tool-call accuracy
Memory quality
Latency
Hallucinations
Safety events

Evaluating RAG Systems

RAG systems should be evaluated for:

Retrieval quality
Relevance
Faithfulness
Grounding accuracy
Citation quality

Evaluating Function-Calling

Organizations should validate:

Correct tool selection
Parameter accuracy
Workflow reliability
Error recovery

Evaluating Conversation Memory

Memory systems should be evaluated for:

Context retention
Consistency
Recall accuracy
Session continuity

Security Considerations

Secure agent systems should implement:

Authentication
Authorization
Managed identities
RBAC
Private networking
Audit logging

Responsible AI Considerations

Organizations should apply:

Safety filters
Guardrails
Human oversight
Content moderation
Usage monitoring

Real-World Scenario

Scenario: Enterprise HR Assistant

Requirements:

Retrieve HR policies
Answer employee questions
Access scheduling systems
Remember user preferences
Escalate sensitive requests

Recommended Design:

RAG using Azure AI Search
Function-calling for HR systems
Stateful conversation memory
Approval workflows for sensitive actions
Grounded response generation

Common AI-103 Exam Tips

Understand Retrieval Concepts

Know:

RAG
Embeddings
Vector search
Hybrid search
Grounding

Learn Function-Calling Concepts

Understand:

Tool schemas
Structured invocation
Tool orchestration
Workflow execution

Understand Memory Systems

Know:

Stateful vs stateless agents
Short-term vs long-term memory
Context management
Vector memory

Understand Agent Orchestration

Know how agents combine:

Retrieval
Tool usage
Memory
Reasoning

Summary

Modern enterprise agents combine:

Retrieval systems
Function-calling
Conversation memory
Workflow orchestration

For the AI-103 exam, you should understand:

RAG architectures
Vector search
Embeddings
Grounding
Function-calling
Tool schemas
Tool orchestration
Stateful memory
Context management
Agent reasoning loops
Monitoring and governance

These concepts are foundational to building scalable and intelligent AI agents with Azure AI Foundry.

Practice Exam Questions

Question 1

What is the primary purpose of Retrieval-Augmented Generation (RAG)?

A. Reduce GPU temperatures
B. Combine retrieval systems with LLM generation
C. Eliminate vector search
D. Replace APIs completely

Answer

B. Combine retrieval systems with LLM generation

Explanation

RAG combines retrieval and generation to improve grounded responses.

Question 2

Why are embeddings important in retrieval systems?

A. They increase firewall security
B. They enable semantic similarity comparisons
C. They replace orchestration engines
D. They remove token limits

Answer

B. They enable semantic similarity comparisons

Explanation

Embeddings support semantic vector search.

Question 3

What is a key advantage of hybrid search?

A. It disables semantic ranking
B. It combines keyword and vector search
C. It removes indexing requirements
D. It eliminates embeddings

Answer

B. It combines keyword and vector search

Explanation

Hybrid search improves retrieval quality by combining approaches.

Question 4

What is the purpose of function-calling in agent systems?

A. Reduce network traffic only
B. Allow models to invoke external tools and services
C. Eliminate APIs
D. Disable workflows

Answer

B. Allow models to invoke external tools and services

Explanation

Function-calling enables interaction with external systems.

Question 5

What information is typically included in a tool schema?

A. GPU temperature metrics
B. Parameters, data types, and outputs
C. Only firewall settings
D. Only vector dimensions

Answer

B. Parameters, data types, and outputs

Explanation

Schemas define structured tool interfaces.

Question 6

Why is conversation memory important?

A. It reduces all storage costs
B. It maintains continuity and context across interactions
C. It removes orchestration needs
D. It disables tool invocation

Answer

B. It maintains continuity and context across interactions

Explanation

Memory improves user experiences and multistep workflows.

Question 7

What is a characteristic of stateful agents?

A. They never store context
B. They maintain conversation history and state
C. They disable retrieval systems
D. They remove prompt engineering

Answer

B. They maintain conversation history and state

Explanation

Stateful agents retain memory across interactions.

Question 8

What is a common challenge when using LLM conversation memory?

A. Unlimited context windows
B. Context window limitations and token constraints
C. Elimination of embeddings
D. Removal of grounding

Answer

B. Context window limitations and token constraints

Explanation

LLMs can process only limited amounts of context.

Question 9

Which Azure service is commonly used for enterprise retrieval in RAG architectures?

A. Azure DevOps
B. Azure AI Search
C. Azure Virtual Desktop
D. Azure Batch

Answer

B. Azure AI Search

Explanation

Azure AI Search supports vector and hybrid search for RAG systems.

Question 10

What should organizations monitor in agent systems?

A. Only GPU fan speeds
B. Retrieval quality, tool usage, memory accuracy, and safety
C. Only prompt lengths
D. Only authentication failures

Answer

B. Retrieval quality, tool usage, memory accuracy, and safety

Explanation

Comprehensive monitoring improves reliability, governance, and user trust.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Artificial Intelligence (AI), Azure AI, Microsoft Certification May 25, 2026

Define agent roles, goals, conversation-tracking approach, and tool schemas (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build agents by using Foundry
      --> Define agent roles, goals, conversation-tracking approach, and tool schemas

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

AI agents are rapidly becoming one of the most important components of modern AI systems.

Unlike basic chatbots, agents can:

Reason through tasks
Maintain context
Use tools
Execute workflows
Coordinate multistep actions
Interact with external systems

Azure AI Foundry provides tools and frameworks for building agentic systems.

For the AI-103: Develop AI Apps and Agents on Azure certification exam, understanding agent design principles is critical.

This topic focuses on:

Agent roles
Agent goals
Conversation tracking
Tool schemas
Tool orchestration
State management
Memory design
Workflow coordination

What Is an AI Agent?

An AI agent is an AI system capable of:

Understanding objectives
Making decisions
Using tools
Maintaining context
Performing actions
Adapting to changing inputs

Agents are more autonomous than standard prompt-response systems.

Characteristics of AI Agents

Agents commonly include:

Reasoning
Planning
Memory
Tool usage
Workflow orchestration
Goal-oriented behavior

Agent Roles

An agent role defines:

The agent’s responsibilities
Behavioral expectations
Scope of operation
Allowed actions

Why Agent Roles Matter

Clearly defined roles help:

Improve consistency
Reduce unsafe behavior
Prevent scope creep
Improve reliability

Examples of Agent Roles

Examples include:

Customer support assistant
Financial analyst
Research assistant
Scheduling coordinator
Coding assistant
IT operations assistant

Specialized vs General-Purpose Agents

Specialized Agents

Focused on narrow tasks.

Benefits:

Higher reliability
Better governance
Easier evaluation

General-Purpose Agents

Handle broad tasks.

Benefits:

Greater flexibility
Wider applicability

Tradeoff:

Increased complexity and risk

Defining Agent Goals

Goals define:

Desired outcomes
Success criteria
Task objectives

Goal-Oriented Design

Good goals are:

Clear
Measurable
Constrained
Actionable

Examples of Agent Goals

Examples include:

Resolve customer tickets
Retrieve accurate company policies
Generate code suggestions
Schedule meetings
Summarize documents

Constraints in Goal Design

Goals should include:

Safety boundaries
Compliance rules
Tool restrictions
Escalation conditions

Agent Instructions and System Prompts

Agents typically receive:

System instructions
Behavioral guidance
Operational constraints

These instructions influence agent behavior.

Conversation Tracking

Conversation tracking maintains:

Dialogue history
User context
Workflow state
Interaction continuity

Why Conversation Tracking Matters

Without conversation tracking:

Agents lose context
Responses become inconsistent
Multistep workflows fail

Short-Term Conversation Memory

Short-term memory may store:

Recent prompts
Recent responses
Current workflow state

Long-Term Memory

Long-term memory may store:

User preferences
Historical interactions
Persistent knowledge

Session State Management

State management tracks:

Current tasks
Workflow progress
Tool outputs
Active context

Stateless vs Stateful Agents

Stateless Agents

Do not retain context between interactions.

Benefits:

Simpler design
Lower storage requirements

Stateful Agents

Maintain conversation history and workflow state.

Benefits:

Better continuity
Improved multistep reasoning

Context Window Management

LLMs have limited context windows.

Applications may need to:

Trim conversation history
Summarize prior interactions
Retrieve external memory

Memory Strategies

Common memory strategies include:

Rolling conversation windows
Summarization memory
Vector memory
Persistent storage

Retrieval-Augmented Memory

Agents may retrieve:

Historical conversations
Knowledge documents
Workflow data

This improves continuity.

Conversation Persistence

Persistent conversation storage may use:

Databases
Search indexes
Vector stores

Tool Usage in Agent Systems

Agents often interact with:

APIs
Databases
Search systems
External applications
Workflow services

What Is a Tool Schema?

A tool schema defines:

Tool name
Purpose
Input parameters
Output structure
Validation rules

Purpose of Tool Schemas

Tool schemas help:

Standardize interactions
Reduce ambiguity
Improve reliability
Enable function calling

Tool Schema Components

Tool schemas commonly include:

Function name
Description
Parameters
Data types
Required fields

Example Tool Schema

Example:

Tool: GetWeather
Inputs:
- City name
- Date
Output:
- Temperature
- Forecast

Structured Tool Invocation

Structured tool schemas allow agents to:

Generate valid requests
Interact predictably with systems
Reduce execution failures

Function Calling

Function calling enables models to:

Invoke external tools
Execute structured operations
Retrieve external data

Tool Selection Logic

Agents may decide:

Whether a tool is needed
Which tool to invoke
How to sequence tool calls

Multi-Tool Workflows

Complex agents may use:

Multiple tools
Sequential workflows
Conditional branching

Tool Access Controls

Organizations may restrict:

Which tools agents can use
When tools can be invoked
Which users may trigger actions

Safety Considerations for Tool Usage

Improper tool usage can:

Leak data
Execute unsafe actions
Cause workflow failures

Human Approval Workflows

Some actions may require:

Human review
Approval checkpoints
Escalation workflows

Agent Planning

Agents may perform:

Task decomposition
Sequential planning
Goal prioritization

Multistep Reasoning

Agents may:

Gather information
Use tools
Analyze results
Generate conclusions

Orchestration Frameworks

Orchestration frameworks coordinate:

Agent logic
Tool execution
Workflow progression
State transitions

Error Handling in Agents

Agents should handle:

Invalid tool outputs
API failures
Missing data
Ambiguous user requests

Monitoring Agent Behavior

Organizations should monitor:

Tool usage
Conversation quality
Safety violations
Goal completion rates

Trace Logging

Trace logs may capture:

Prompt sequences
Tool calls
Workflow decisions
Agent reasoning steps

Evaluation of Agent Systems

Organizations should evaluate:

Goal completion
Accuracy
Relevance
Safety
Tool reliability

Governance and Compliance

Enterprise agent systems may require:

Access controls
Audit logging
Compliance policies
Responsible AI governance

Real-World Scenario

Scenario: Enterprise IT Support Agent

Requirements:

Resolve common IT requests
Access ticketing systems
Maintain user context
Escalate high-risk actions

Recommended Design:

Specialized support role
Defined goals
Stateful conversation tracking
Structured tool schemas
Human approval workflows

Common AI-103 Exam Tips

Understand Agent Roles

Know:

Specialized vs general-purpose agents
Role boundaries
Behavioral constraints

Learn Conversation Tracking Concepts

Understand:

Stateful vs stateless agents
Memory approaches
Context management

Understand Tool Schemas

Know:

Function definitions
Parameters
Structured tool invocation
Function calling

Learn Governance Concepts

Understand:

Tool access controls
Human approvals
Audit logging
Safety constraints

Summary

Agent design is a core part of modern AI systems.

For the AI-103 exam, you should understand:

Agent roles
Goal-oriented behavior
Conversation tracking
Memory management
Stateful workflows
Tool schemas
Function calling
Tool orchestration
Workflow planning
Safety controls
Human approvals
Monitoring and governance

These concepts are foundational for building secure, scalable, and reliable agentic systems using Azure AI Foundry.

Practice Exam Questions

Question 1

What is the primary purpose of an agent role?

A. Increase GPU utilization
B. Define responsibilities and behavioral boundaries
C. Eliminate tool usage
D. Remove workflow orchestration

Answer

B. Define responsibilities and behavioral boundaries

Explanation

Agent roles establish scope, expectations, and operational constraints.

Question 2

Why are clearly defined agent goals important?

A. They eliminate monitoring
B. They provide measurable objectives and task direction
C. They reduce storage requirements only
D. They remove authentication needs

Answer

B. They provide measurable objectives and task direction

Explanation

Goals help agents focus on desired outcomes.

Question 3

What is the purpose of conversation tracking?

A. Increase vector dimensions
B. Maintain context and workflow continuity
C. Disable memory systems
D. Remove APIs

Answer

B. Maintain context and workflow continuity

Explanation

Conversation tracking preserves interaction history and state.

Question 4

What is a key benefit of stateful agents?

A. They avoid all storage requirements
B. They maintain continuity across interactions
C. They eliminate workflows
D. They remove tool schemas

Answer

B. They maintain continuity across interactions

Explanation

Stateful agents retain memory and conversation context.

Question 5

What is a tool schema?

A. A GPU optimization technique
B. A structured definition of tool inputs and outputs
C. A firewall policy
D. A token compression method

Answer

B. A structured definition of tool inputs and outputs

Explanation

Tool schemas standardize external tool interactions.

Question 6

What is the purpose of function calling?

A. Eliminate orchestration
B. Allow models to invoke external tools dynamically
C. Replace APIs entirely
D. Remove authentication

Answer

B. Allow models to invoke external tools dynamically

Explanation

Function calling enables structured tool execution.

Question 7

Why are tool access controls important?

A. They reduce GPU memory usage
B. They restrict unsafe or unauthorized tool usage
C. They eliminate monitoring
D. They disable workflows

Answer

B. They restrict unsafe or unauthorized tool usage

Explanation

Access controls improve safety and governance.

Question 8

What is a common challenge with large conversation histories?

A. Unlimited context windows
B. Context window limitations in LLMs
C. Elimination of memory usage
D. Reduced orchestration complexity

Answer

B. Context window limitations in LLMs

Explanation

LLMs can only process limited amounts of context.

Question 9

What is the purpose of human approval workflows?

A. Increase hallucinations
B. Provide oversight for sensitive or high-risk actions
C. Remove governance requirements
D. Disable trace logging

Answer

B. Provide oversight for sensitive or high-risk actions

Explanation

Human review reduces operational risk.

Question 10

What should organizations monitor in agent systems?

A. Only GPU temperatures
B. Tool usage, safety, conversation quality, and task completion
C. Only token counts
D. Only API latency

Answer

B. Tool usage, safety, conversation quality, and task completion

Explanation

Comprehensive monitoring improves reliability and governance.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Artificial Intelligence (AI), Azure AI, Generative AI, Microsoft Certification May 25, 2026

Configure an application to connect to a Foundry project (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build generative applications by using Foundry
      --> Configure an application to connect to a Foundry project

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Azure AI Foundry provides a centralized environment for developing, deploying, and managing AI applications and agentic solutions.

Applications that use generative AI models, agents, retrieval systems, or multimodal capabilities must connect securely and reliably to Foundry projects.

This topic is important for the AI-103: Develop AI Apps and Agents on Azure certification exam.

For the AI-103 exam, you should understand:

Azure AI Foundry projects
Application connectivity
Authentication methods
SDK configuration
Endpoint configuration
Deployment configuration
Managed identities
API keys
Environment variables
Network security
Role-based access control (RBAC)
Connecting to deployed models and agents
Configuration management
Monitoring and troubleshooting

What Is an Azure AI Foundry Project?

An Azure AI Foundry project is a centralized workspace used to:

Manage AI resources
Deploy models
Configure agents
Build workflows
Store evaluation assets
Monitor AI systems

Projects help organize AI development and operations.

Components of a Foundry Project

A Foundry project may include:

Model deployments
Agent configurations
Prompt flows
Evaluation datasets
Connections
Search resources
Storage resources
Monitoring tools

Why Applications Need Project Connectivity

Applications connect to Foundry projects to:

Access deployed models
Invoke agents
Perform retrieval operations
Execute workflows
Use AI services securely

Common Connection Scenarios

Applications commonly connect to:

Chat models
Embedding models
Multimodal models
Agent services
Prompt flow endpoints
Azure AI Search resources

Connection Architecture

Typical connectivity includes:

Application
Authentication layer
Foundry project endpoint
Model or agent deployment

SDK-Based Connectivity

Applications often use SDKs to:

Authenticate
Send prompts
Receive responses
Stream outputs
Manage workflows

SDKs simplify development.

API-Based Connectivity

Applications may also use:

REST APIs
HTTP endpoints
Direct service requests

Authentication Methods

Applications must authenticate securely.

Common methods include:

API keys
Managed identities
Azure Active Directory (Azure AD)
Keyless authentication

API Key Authentication

API keys are:

Simple to configure
Easy for development and testing

However, they require secure storage.

Managed Identity Authentication

Managed identities provide:

Secretless authentication
Improved security
Automatic credential management

Managed identity is recommended for production workloads.

Azure AD Authentication

Azure AD enables:

Enterprise identity management
Role-based access
Secure authentication workflows

Keyless Authentication

Keyless authentication reduces:

Credential exposure
Secret management overhead

Secure Credential Storage

Applications should avoid:

Hardcoded secrets
Plain-text credentials

Credentials should be stored securely.

Environment Variables

Environment variables commonly store:

API endpoints
Deployment names
Keys
Configuration settings

Configuration Files

Applications may use:

JSON configuration files
YAML files
Application settings

Endpoint Configuration

Applications must connect to the correct:

Foundry endpoint
Model deployment endpoint
Agent endpoint

Deployment Names

Applications typically reference:

Specific deployment names
Model identifiers
Agent identifiers

Connecting to Model Deployments

Applications may connect to:

Chat completion models
Embedding models
Code models
Multimodal models

Connecting to Agent Workflows

Applications may invoke agents that:

Use tools
Access memory
Execute workflows
Coordinate tasks

Connecting to Prompt Flows

Applications can invoke:

Prompt flow endpoints
Orchestrated workflows
Multi-step pipelines

Connecting to Azure AI Search

RAG applications often connect to:

Azure AI Search
Vector indexes
Semantic search pipelines

Role-Based Access Control (RBAC)

RBAC controls:

Resource permissions
Service access
Administrative privileges

Least Privilege Principle

Applications should receive:

Only required permissions
Minimal access rights

Private Networking

Organizations may secure connectivity using:

Private endpoints
Virtual networks
Network isolation

Firewall Configuration

Firewall rules may restrict:

Public access
Unauthorized IP ranges

Secure Communication

Applications should use:

HTTPS
Encrypted communication
Secure APIs

SDK Initialization

Applications typically initialize:

Client objects
Authentication providers
Connection settings

Client Configuration

Client configuration may include:

Endpoint URLs
API versions
Deployment names
Authentication credentials

Streaming Configuration

Applications may enable:

Streaming responses
Incremental output rendering

Retry Policies

Applications should implement:

Retry logic
Exponential backoff
Timeout handling

Error Handling

Applications should handle:

Authentication failures
Network issues
Rate limits
Invalid requests

Logging and Monitoring

Applications should log:

Requests
Responses
Failures
Latency metrics

Observability

Observability helps organizations:

Monitor usage
Diagnose issues
Improve reliability

Application Scalability

Applications should support:

High concurrency
Distributed workloads
Elastic scaling

Cost Considerations

Connection design impacts:

Token usage
API consumption
Search operations
Infrastructure costs

CI/CD Integration

Connection settings may be managed through:

Deployment pipelines
Infrastructure as code
Environment promotion

Development vs Production Environments

Organizations often separate:

Development
Testing
Staging
Production

Each environment may use different:

Endpoints
Credentials
Policies

Multi-Region Connectivity

Global applications may connect to:

Multiple regional deployments
Regional failover systems

High Availability

Applications should support:

Redundant deployments
Failover strategies
Resilient architecture

Governance Considerations

Organizations may enforce:

Access policies
Security baselines
Audit logging
Compliance requirements

Troubleshooting Connectivity Issues

Common issues include:

Invalid credentials
Incorrect endpoints
Missing RBAC permissions
Network restrictions
Deployment mismatches

Performance Optimization

Organizations should optimize:

Connection reuse
Latency
Request batching
Streaming efficiency

Real-World Scenario

Scenario: Enterprise AI Assistant

Requirements:

Secure authentication
RAG integration
Agent orchestration
Enterprise access control

Recommended Approach:

Managed identity
RBAC
Private networking
Azure AI Search integration
SDK-based connectivity

Common AI-103 Exam Tips

Understand Authentication Options

Know when to use:

API keys
Managed identities
Azure AD

Understand Endpoint Configuration

Know:

Deployment names
Service endpoints
Agent endpoints

Learn RBAC Concepts

Understand:

Least privilege
Role assignments
Secure access management

Understand Networking Concepts

Know:

Private endpoints
Firewalls
Secure connectivity

Learn Application Integration Concepts

Understand:

SDK initialization
Client configuration
Retry logic
Monitoring

Summary

Connecting applications to Azure AI Foundry projects is a foundational skill for AI-103.

For the exam, you should understand:

Foundry projects
Application connectivity
SDK integration
API integration
Authentication methods
Managed identities
RBAC
Deployment configuration
Endpoint management
Networking security
Logging and monitoring
Scalability and reliability

These skills are essential for building secure, scalable enterprise AI applications on Azure.

Practice Exam Questions

Question 1

What is the purpose of an Azure AI Foundry project?

A. Replace Azure subscriptions
B. Centrally manage AI resources, deployments, and workflows
C. Eliminate authentication
D. Replace APIs entirely

Answer

B. Centrally manage AI resources, deployments, and workflows

Explanation

Foundry projects organize AI development and operational assets.

Question 2

Which authentication method is recommended for production Azure workloads?

A. Hardcoded credentials
B. Managed identity
C. Shared public keys
D. Anonymous access

Answer

B. Managed identity

Explanation

Managed identities improve security by avoiding embedded secrets.

Question 3

What is a primary advantage of SDKs?

A. They eliminate APIs completely
B. They simplify application development and integration
C. They remove all authentication requirements
D. They prevent monitoring

Answer

B. They simplify application development and integration

Explanation

SDKs provide abstractions that simplify connectivity and workflow development.

Question 4

Why should applications use environment variables?

A. To increase GPU performance
B. To securely manage configuration values
C. To eliminate authentication
D. To disable RBAC

Answer

B. To securely manage configuration values

Explanation

Environment variables help manage endpoints and credentials securely.

Question 5

What does RBAC primarily control?

A. Token compression
B. Permissions and access to resources
C. Model quantization
D. Network bandwidth

Answer

B. Permissions and access to resources

Explanation

RBAC enforces authorization policies.

Question 6

Why are private endpoints used?

A. To increase hallucinations
B. To improve network security and isolate traffic
C. To disable monitoring
D. To reduce embedding dimensions

Answer

B. To improve network security and isolate traffic

Explanation

Private endpoints help secure enterprise AI workloads.

Question 7

What is commonly required when connecting to a deployed model?

A. Deployment name
B. Firewall removal
C. Disabling authentication
D. Public anonymous access

Answer

A. Deployment name

Explanation

Applications typically reference deployment identifiers.

Question 8

Why should applications implement retry policies?

A. To increase hallucinations
B. To recover from transient failures and improve reliability
C. To disable APIs
D. To remove authentication

Answer

B. To recover from transient failures and improve reliability

Explanation

Retry logic improves resiliency.

Question 9

Which service is commonly integrated for RAG search functionality?

A. Azure AI Search
B. Azure DNS
C. Azure Backup
D. Azure Batch

Answer

A. Azure AI Search

Explanation

Azure AI Search supports vector and semantic retrieval.

Question 10

What is the least privilege principle?

A. Give all users full access
B. Grant only the permissions necessary to perform required tasks
C. Disable RBAC
D. Allow anonymous authentication

Answer

B. Grant only the permissions necessary to perform required tasks

Explanation

Least privilege reduces security risk by minimizing unnecessary permissions.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Azure AI, Generative AI, Microsoft Certification May 25, 2026

Integrate generative workflows into applications by using Foundry SDKs and connectors (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build generative applications by using Foundry
      --> Integrate generative workflows into applications by using Foundry SDKs and connectors

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI applications rarely operate in isolation.

Enterprise generative AI solutions typically integrate with:

Web applications
APIs
Databases
Search systems
Business applications
Workflow engines
External tools

Azure AI Foundry provides:

SDKs
APIs
Connectors
Agent frameworks
Workflow orchestration capabilities

These services help developers integrate generative AI into enterprise applications.

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of integrating generative workflows into applications.

For the AI-103 exam, you should understand:

Foundry SDKs
APIs
Connectors
Workflow orchestration
Tool integration
Agent integration
RAG integration
Authentication
Deployment integration
Event-driven workflows
Monitoring and governance

What Are Foundry SDKs?

SDKs (Software Development Kits) provide:

Libraries
APIs
Helper functions
Authentication support
Workflow integration tools

SDKs simplify application development.

Benefits of SDKs

SDKs help developers:

Reduce development complexity
Standardize integration
Accelerate deployment
Improve reliability

Common SDK Capabilities

SDKs commonly support:

Model invocation
Agent orchestration
Function calling
Authentication
Streaming responses
Workflow management
Monitoring integration

APIs vs SDKs

APIs

Provide direct service access.

SDKs

Provide higher-level development abstractions.

SDKs often simplify API usage.

What Are Connectors?

Connectors integrate AI systems with:

External services
Enterprise applications
Data sources
Workflow systems

Common Connector Scenarios

Examples include:

CRM integration
ERP integration
SharePoint access
Database connectivity
Messaging systems
Search services

Workflow Integration

Generative workflows may integrate with:

Web applications
Mobile applications
Enterprise platforms
Automation systems

Web Application Integration

Generative AI commonly integrates into:

Chat interfaces
Copilots
Knowledge assistants
Recommendation systems

API-Based Integration

Applications often communicate with AI systems through:

REST APIs
HTTP endpoints
SDK abstractions

Authentication and Authorization

Secure integration requires:

Authentication
Authorization
Identity management

Managed Identity

Managed identities allow Azure services to:

Authenticate securely
Avoid hardcoded secrets
Access resources safely

Keyless Authentication

Keyless authentication improves security by reducing:

API key exposure
Credential management complexity

Secure Credential Storage

Applications should protect:

API keys
Tokens
Connection strings

Role-Based Access Control (RBAC)

RBAC helps control:

Resource permissions
Service access
Administrative privileges

Event-Driven Workflows

Event-driven systems react to:

User actions
File uploads
Database changes
External events

Asynchronous Workflows

Asynchronous workflows:

Improve scalability
Reduce blocking operations
Support long-running tasks

Streaming Responses

Streaming enables applications to:

Display responses incrementally
Improve user experience
Reduce perceived latency

Conversational Application Integration

Conversational systems often integrate:

Memory
Retrieval
Tool usage
User context

Integrating Retrieval-Augmented Generation (RAG)

RAG integration typically includes:

Vector search
Embedding generation
Retrieval pipelines
Prompt grounding

Azure AI Search Integration

Applications commonly integrate Azure AI Search for:

Vector search
Semantic search
Hybrid retrieval

Tool-Augmented Integration

Applications may integrate tools such as:

Databases
Search APIs
Business systems
External APIs

Function Calling Integration

Function calling enables:

Dynamic tool invocation
Structured interactions
Workflow orchestration

Agent Integration

Agent-based systems may:

Coordinate tools
Perform multistep reasoning
Execute workflows
Manage task state

Workflow Orchestration

Workflow orchestration coordinates:

AI reasoning
Tool execution
Retrieval
Human approvals

State Management

Integrated systems often maintain:

Session state
Workflow progress
User context

Memory Integration

Applications may integrate:

Short-term memory
Long-term memory
User preferences

Human-in-the-Loop Integration

Enterprise applications may require:

Human approvals
Review workflows
Escalation paths

Monitoring Integration

Applications should integrate monitoring for:

Errors
Latency
Tool usage
Costs
Safety violations

Logging and Traceability

Logging supports:

Troubleshooting
Auditing
Workflow analysis
Compliance

Trace Logging

Trace logs may capture:

Prompt flows
Tool calls
Retrieval steps
Workflow execution

Error Handling

Applications should handle:

API failures
Timeout errors
Invalid responses
Authentication failures

Retry Mechanisms

Retry strategies improve reliability by:

Recovering from transient failures
Reducing workflow interruptions

Scalability Considerations

Integrated AI systems should support:

High concurrency
Dynamic scaling
Distributed workloads

Latency Considerations

Developers should optimize:

Retrieval speed
Tool invocation times
Model response times

Cost Optimization

Organizations should optimize:

Token usage
API calls
Search operations
Infrastructure costs

CI/CD Integration

Generative AI applications may integrate with:

Automated deployment pipelines
Testing frameworks
Infrastructure automation

Testing Integrated Workflows

Organizations should test:

Workflow correctness
Tool integration
Retrieval quality
Safety compliance

Safety Integration

Applications should integrate:

Content filtering
Safety policies
Guardrails
Approval workflows

Governance and Compliance

Enterprise systems may require:

Audit logging
Data protection
Regulatory compliance
Access controls

Azure AI Foundry Integration Features

Azure AI Foundry supports:

SDK-based development
Workflow orchestration
Model deployment
Agent development
Evaluation pipelines
Monitoring

Real-World Integration Scenarios

Scenario 1: Enterprise Knowledge Assistant

Requirements:

Document retrieval
Conversational AI
Enterprise search integration

Recommended Integration:

Foundry SDK + Azure AI Search

Scenario 2: Customer Support Copilot

Requirements:

CRM integration
Ticket lookup
Escalation workflows

Recommended Integration:

Tool-augmented agent workflows

Scenario 3: Financial Workflow Automation

Requirements:

Human approvals
Audit logging
Secure authentication

Recommended Integration:

HITL workflow + RBAC + trace logging

Scenario 4: AI Research Assistant

Requirements:

Multistep reasoning
Web search integration
Citation generation

Recommended Integration:

RAG + orchestration workflows

Common AI-103 Exam Tips

Understand SDK vs API Differences

Know:

SDK abstractions
API integrations
Authentication approaches

Learn Connector Concepts

Understand:

External integrations
Enterprise systems
Workflow connectors

Understand Workflow Integration

Know:

Tool orchestration
Agent integration
Event-driven workflows
Streaming responses

Learn Security Concepts

Understand:

Managed identity
Keyless credentials
RBAC
Secure secret handling

Summary

Modern generative AI systems depend heavily on integration.

For the AI-103 exam, you should understand:

Foundry SDKs
APIs
Connectors
Workflow orchestration
Function calling
Agent integration
RAG integration
Authentication and RBAC
Event-driven workflows
Monitoring and logging
CI/CD integration
Governance and compliance

These concepts are foundational for building scalable enterprise AI applications and agentic systems on Azure.

Practice Exam Questions

Question 1

What is the primary purpose of an SDK?

A. Replace APIs entirely
B. Simplify application development using libraries and abstractions
C. Eliminate authentication requirements
D. Disable workflow orchestration

Answer

B. Simplify application development using libraries and abstractions

Explanation

SDKs provide tools and abstractions that simplify development.

Question 2

What is a connector in a generative AI solution?

A. A GPU optimization engine
B. A mechanism for integrating external systems and services
C. A vector compression method
D. A storage replication service

Answer

B. A mechanism for integrating external systems and services

Explanation

Connectors enable integration with business applications and data sources.

Question 3

Why are managed identities important?

A. They increase token limits
B. They provide secure authentication without hardcoded credentials
C. They replace vector search
D. They eliminate RBAC

Answer

B. They provide secure authentication without hardcoded credentials

Explanation

Managed identities improve security by avoiding embedded secrets.

Question 4

What is the benefit of streaming responses?

A. Eliminates all latency
B. Improves user experience by displaying incremental output
C. Disables monitoring
D. Prevents tool invocation

Answer

B. Improves user experience by displaying incremental output

Explanation

Streaming responses reduce perceived latency.

Question 5

What is the purpose of function calling?

A. Compress prompts
B. Allow models to invoke external tools dynamically
C. Replace orchestration
D. Eliminate APIs

Answer

B. Allow models to invoke external tools dynamically

Explanation

Function calling enables structured tool interactions.

Question 6

Which Azure service is commonly integrated for vector and semantic search?

A. Azure AI Search
B. Azure DNS
C. Azure Backup
D. Azure Batch

Answer

A. Azure AI Search

Explanation

Azure AI Search supports vector and semantic retrieval.

Question 7

What is a key advantage of asynchronous workflows?

A. Increased blocking operations
B. Improved scalability and support for long-running tasks
C. Removal of authentication
D. Elimination of APIs

Answer

B. Improved scalability and support for long-running tasks

Explanation

Asynchronous workflows support efficient distributed execution.

Question 8

Why is trace logging important?

A. It removes monitoring requirements
B. It provides visibility into workflow execution and troubleshooting
C. It disables retrieval pipelines
D. It eliminates RBAC

Answer

B. It provides visibility into workflow execution and troubleshooting

Explanation

Trace logs help monitor workflows and investigate issues.

Question 9

What is the purpose of RBAC?

A. Increase vector dimensions
B. Control permissions and access to resources
C. Replace authentication
D. Reduce prompt sizes

Answer

B. Control permissions and access to resources

Explanation

RBAC enforces authorization policies.

Question 10

What is a major challenge when integrating complex generative workflows?

A. Eliminating all costs
B. Managing latency, scalability, and reliability
C. Removing all monitoring
D. Disabling orchestration

Answer

B. Managing latency, scalability, and reliability

Explanation

Integrated workflows often involve multiple services and asynchronous operations.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Artificial Intelligence (AI), Azure AI, Microsoft Certification May 25, 2026

Evaluate models and apps, including detecting fabrications, relevance, quality, and safety (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build generative applications by using Foundry
      --> Evaluate models and apps, including detecting fabrications, relevance, quality, and safety

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Building generative AI applications is only part of the development process.

Organizations must also evaluate whether AI systems are:

Accurate
Reliable
Relevant
Safe
Grounded
Trustworthy

AI systems can generate:

Hallucinations
Unsafe content
Biased responses
Irrelevant answers
Inconsistent outputs

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of evaluating models and applications.

For the AI-103 exam, you should understand:

Model evaluation
Application evaluation
Fabrication detection
Groundedness
Relevance evaluation
Quality evaluation
Safety evaluation
Responsible AI testing
Automated evaluators
Human evaluation
Benchmarking
Monitoring and continuous evaluation

Why AI Evaluation Matters

Evaluation is essential because generative AI systems are probabilistic.

This means:

Responses may vary
Outputs may be incorrect
Safety risks may occur
Hallucinations may appear

Without evaluation, organizations cannot reliably trust AI systems.

What Is AI Evaluation?

AI evaluation is the process of measuring:

Accuracy
Safety
Reliability
Relevance
Groundedness
User satisfaction

Types of AI Evaluation

Common evaluation categories include:

Model evaluation
Prompt evaluation
Retrieval evaluation
Application evaluation
Safety evaluation
Human evaluation

Model Evaluation

Model evaluation focuses on:

Model quality
Accuracy
Performance
Reasoning ability

Application Evaluation

Application evaluation measures:

End-to-end user experience
Workflow success
Tool orchestration quality
Groundedness

What Are Fabrications?

Fabrications are generated outputs that:

Are incorrect
Are unsupported
Contain invented facts
Misrepresent information

Fabrications are commonly called hallucinations.

Causes of Fabrications

Fabrications may occur because:

The model lacks relevant knowledge
Prompts are ambiguous
Retrieval quality is poor
Context is insufficient
Safety constraints are weak

Fabrication Detection

Organizations should evaluate whether outputs:

Match trusted sources
Remain grounded
Avoid unsupported claims

Groundedness Evaluation

Groundedness measures whether responses are supported by:

Retrieved documents
Enterprise data
Trusted sources

Importance of Groundedness

Grounded responses:

Improve trust
Reduce hallucinations
Increase explainability

Retrieval Quality Evaluation

RAG systems should evaluate:

Search relevance
Retrieved chunk quality
Citation accuracy
Context completeness

Relevance Evaluation

Relevance measures whether responses:

Answer the user’s question
Stay on-topic
Match user intent

Quality Evaluation

Quality evaluations may assess:

Clarity
Completeness
Coherence
Fluency
Professionalism

Consistency Evaluation

Consistency measures whether models:

Produce stable responses
Avoid contradictory outputs
Maintain predictable behavior

Safety Evaluation

Safety evaluations identify:

Harmful outputs
Toxic content
Unsafe instructions
Policy violations

Responsible AI Evaluation

Responsible AI testing focuses on:

Fairness
Safety
Transparency
Accountability
Privacy

Bias Evaluation

Organizations should evaluate whether models:

Produce biased outputs
Treat groups unfairly
Reinforce stereotypes

Toxicity Detection

Toxicity evaluations identify:

Offensive language
Hate speech
Harassment
Abusive content

Jailbreak Testing

Jailbreak testing evaluates whether users can bypass:

Safety controls
Content filters
Guardrails

Adversarial Testing

Adversarial testing intentionally challenges models using:

Malicious prompts
Edge cases
Prompt injection attacks

Prompt Injection Testing

Prompt injection testing evaluates whether:

External content manipulates model behavior
Instructions override safety policies

Automated Evaluators

Automated evaluators use:

Rules
Scoring systems
AI-based evaluators

To assess model outputs.

AI-Assisted Evaluation

Some systems use LLMs to evaluate:

Relevance
Groundedness
Quality
Safety

Human Evaluation

Human reviewers may evaluate:

Accuracy
Tone
Helpfulness
Safety
Business alignment

Human-in-the-Loop Evaluation

Human-in-the-loop evaluation combines:

Automated evaluation
Human oversight
Expert validation

Benchmarking Models

Benchmarking compares models using:

Standard datasets
Consistent prompts
Defined metrics

A/B Testing

A/B testing compares:

Different prompts
Different models
Different workflows

Evaluation Metrics

Common metrics include:

Precision
Recall
Accuracy
Relevance
Groundedness
Toxicity scores
Latency
User satisfaction

Precision and Recall

Precision

Measures how many retrieved results are relevant.

Recall

Measures how many relevant results were successfully retrieved.

Latency Evaluation

Organizations should measure:

Response times
Retrieval delays
Tool execution times

Cost Evaluation

Cost evaluation considers:

Token usage
API calls
Infrastructure consumption

User Satisfaction Evaluation

Organizations may measure:

User feedback
Completion success
Satisfaction ratings

Continuous Evaluation

AI systems should be evaluated continuously because:

User behavior changes
Data evolves
Model drift may occur

Model Drift

Model drift occurs when:

Performance changes over time
Inputs evolve
User expectations shift

Monitoring Production Systems

Organizations should monitor:

Safety violations
Hallucination rates
Retrieval failures
Latency spikes
Cost increases

Evaluation Pipelines

Evaluation pipelines automate:

Testing
Scoring
Reporting
Regression analysis

Regression Testing

Regression testing ensures updates do not:

Reduce quality
Break workflows
Increase hallucinations

Azure AI Foundry Evaluation Capabilities

Azure AI Foundry supports:

Evaluation workflows
Automated evaluators
Safety monitoring
Groundedness evaluation
Prompt testing
Trace analysis

Trace Analysis

Trace analysis helps inspect:

Tool calls
Retrieval steps
Agent decisions
Workflow execution

Evaluation Datasets

Organizations should create datasets containing:

Expected outputs
Edge cases
Adversarial prompts
Real-world scenarios

Synthetic Test Data

Synthetic data may help test:

Rare scenarios
Adversarial prompts
Safety boundaries

Real-World Evaluation Scenarios

Scenario 1: Enterprise Chatbot

Requirements:

Accurate responses
Citation support
Low hallucination rate

Recommended Evaluation:

Groundedness testing
Retrieval quality evaluation

Scenario 2: Financial Assistant

Requirements:

High accuracy
Safety compliance
Low fabrication risk

Recommended Evaluation:

Human review
Adversarial testing
Approval workflows

Scenario 3: Customer Support Copilot

Requirements:

Relevant responses
Fast response times
Consistent tone

Recommended Evaluation:

Latency evaluation
Quality scoring
A/B testing

Scenario 4: Agentic Workflow System

Requirements:

Tool accuracy
Safe tool execution
Workflow traceability

Recommended Evaluation:

Trace analysis
Tool execution monitoring
HITL evaluation

Common AI-103 Exam Tips

Understand Evaluation Categories

Know the differences between:

Relevance
Quality
Groundedness
Safety
Consistency

Learn Fabrication Detection Concepts

Understand:

Hallucinations
Unsupported claims
Grounding validation

Understand Safety Testing

Know:

Toxicity testing
Jailbreak testing
Prompt injection evaluation
Adversarial testing

Learn Monitoring Concepts

Understand:

Continuous evaluation
Drift detection
Trace analysis
Regression testing

Summary

Evaluating generative AI systems is critical for building:

Reliable
Safe
Grounded
Trustworthy applications

For the AI-103 exam, you should understand:

Fabrication detection
Groundedness evaluation
Retrieval quality
Relevance testing
Quality evaluation
Safety evaluation
Toxicity detection
Adversarial testing
Human evaluation
Automated evaluators
Monitoring and drift detection
Evaluation pipelines

These concepts are foundational for developing enterprise-grade AI applications and agentic systems on Azure.

Practice Exam Questions

Question 1

What is a fabrication in generative AI?

A. A storage replication process
B. An unsupported or invented response
C. A vector indexing method
D. A deployment strategy

Answer

B. An unsupported or invented response

Explanation

Fabrications, also called hallucinations, are incorrect or invented outputs.

Question 2

What does groundedness measure?

A. GPU performance
B. Whether outputs are supported by trusted sources
C. Network bandwidth
D. Token compression efficiency

Answer

B. Whether outputs are supported by trusted sources

Explanation

Groundedness evaluates factual support from retrieved or trusted data.

Question 3

Which evaluation type focuses on harmful or unsafe outputs?

A. Latency evaluation
B. Safety evaluation
C. Compression evaluation
D. Replication evaluation

Answer

B. Safety evaluation

Explanation

Safety evaluations detect harmful, toxic, or policy-violating outputs.

Question 4

What is the purpose of retrieval quality evaluation in RAG systems?

A. Measure GPU speed
B. Assess search relevance and retrieved context quality
C. Reduce storage redundancy
D. Disable embeddings

Answer

B. Assess search relevance and retrieved context quality

Explanation

Retrieval quality measures how useful and relevant retrieved information is.

Question 5

What is jailbreak testing?

A. Testing storage failures
B. Evaluating attempts to bypass safety controls
C. Measuring retrieval latency
D. Compressing prompts

Answer

B. Evaluating attempts to bypass safety controls

Explanation

Jailbreak testing checks whether users can circumvent AI safety mechanisms.

Question 6

Which metric measures whether responses answer the user’s question appropriately?

A. Relevance
B. Replication
C. Throughput
D. Compression

Answer

A. Relevance

Explanation

Relevance evaluates how well outputs match user intent.

Question 7

Why is continuous evaluation important?

A. To eliminate all infrastructure costs
B. Because models and data can change over time
C. To remove all safety policies
D. To disable monitoring

Answer

B. Because models and data can change over time

Explanation

Continuous evaluation helps detect drift and performance degradation.

Question 8

What is adversarial testing?

A. Testing network redundancy
B. Challenging AI systems with malicious or difficult prompts
C. Increasing vector dimensions
D. Optimizing GPU allocation

Answer

B. Challenging AI systems with malicious or difficult prompts

Explanation

Adversarial testing identifies vulnerabilities and unsafe behaviors.

Question 9

What is a benefit of A/B testing in AI systems?

A. Eliminates monitoring requirements
B. Compares prompts or models to identify better performance
C. Removes the need for evaluation datasets
D. Disables retrieval pipelines

Answer

B. Compares prompts or models to identify better performance

Explanation

A/B testing helps optimize prompts, workflows, and models.

Question 10

Which Azure capability helps inspect workflow execution and tool calls?

A. Trace analysis
B. DNS failover
C. Storage mirroring
D. GPU partitioning

Answer

A. Trace analysis

Explanation

Trace analysis provides visibility into workflow execution and reasoning steps.

Go to the AI-103 Exam Prep Hub main page

Agentic AI, AI, AI-103, Artificial Intelligence (AI), Azure AI, Generative AI, Microsoft Certification May 25, 2026

Design workflows, tool-augmented flows, and multistep reasoning pipelines (AI-103 Exam Prep)

This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Implement generative AI and agentic solutions (30–35%)
   --> Build generative applications by using Foundry
      --> Design workflows, tool-augmented flows, and multistep reasoning pipelines

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI systems are evolving beyond simple prompt-response interactions.

Today’s generative AI applications often:

Use external tools
Perform multistep reasoning
Orchestrate workflows
Retrieve enterprise data
Execute actions autonomously
Coordinate across services

These systems are commonly called:

Agentic systems
Tool-augmented AI systems
AI workflow pipelines

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of designing intelligent workflows and reasoning pipelines.

For the AI-103 exam, you should understand:

AI workflows
Agent orchestration
Tool augmentation
Function calling
Multistep reasoning
Workflow pipelines
Retrieval integration
Memory integration
Planning and execution
Human-in-the-loop workflows
Monitoring and governance

What Are AI Workflows?

AI workflows are structured sequences of operations that combine:

AI reasoning
Data retrieval
Tool execution
Decision-making
Automation

Workflows coordinate multiple steps to complete complex tasks.

Why AI Workflows Matter

Simple prompts are often insufficient for:

Enterprise automation
Complex reasoning
Dynamic decision-making
Multi-system integration

Workflows allow AI systems to:

Break problems into steps
Use external tools
Validate outputs
Iterate toward solutions

What Is Tool Augmentation?

Tool augmentation allows AI systems to use external capabilities.

Examples include:

APIs
Databases
Search engines
Calculators
Business systems
Code interpreters

Why Tool Augmentation Is Important

Language models alone:

Cannot access real-time data
Cannot execute business actions directly
Cannot reliably perform all calculations

Tools extend AI capabilities.

Common Tool-Augmented Scenarios

Examples include:

Checking inventory
Booking appointments
Querying databases
Sending emails
Executing workflows
Calling REST APIs

What Is Function Calling?

Function calling enables models to:

Detect when a tool is needed
Generate structured tool requests
Invoke external services
Process returned results

Function Calling Workflow

Typical flow:

User submits request
Model determines tool requirement
Model generates function call
External tool executes
Results return to model
Model generates final response

Structured Tool Inputs

Function calling typically uses:

JSON schemas
Structured parameters
Validated inputs

This improves reliability.

Tool Selection

Agentic systems may dynamically choose:

Which tools to use
Which workflows to invoke
Which retrieval strategies to apply

Tool Orchestration

Tool orchestration coordinates multiple tools within a workflow.

Examples include:

Retrieval + summarization
Search + booking systems
Database queries + reporting

Sequential Workflows

Sequential workflows execute steps in order.

Example:

Retrieve customer data
Analyze account status
Generate recommendations
Send response

Parallel Workflows

Parallel workflows execute multiple tasks simultaneously.

Benefits include:

Faster execution
Better scalability
Reduced latency

Conditional Workflows

Conditional workflows branch based on:

User intent
Retrieved data
Safety evaluations
Confidence scores

What Is Multistep Reasoning?

Multistep reasoning breaks complex problems into smaller steps.

This improves:

Accuracy
Planning
Decision quality

Examples of Multistep Reasoning

Examples include:

Research workflows
Financial analysis
Travel planning
Technical troubleshooting

Chain-of-Thought Reasoning

Chain-of-thought reasoning encourages models to:

Reason step-by-step
Decompose problems
Validate intermediate steps

Planning and Execution Models

Agentic systems often separate:

Planning
Execution

The planner decides:

What steps are needed
Which tools to use

The executor performs actions.

Planner-Executor Architectures

Planner-executor architectures support:

Dynamic workflows
Adaptive reasoning
Task decomposition

ReAct Pattern

The ReAct (Reason + Act) pattern combines:

Reasoning
Tool usage
Observation
Iterative decision-making

Reflection and Self-Correction

Some systems support:

Self-evaluation
Output refinement
Error correction

Retrieval-Augmented Workflows

Workflows often integrate:

Vector search
RAG pipelines
Enterprise grounding

Memory in Agentic Systems

AI systems may use memory for:

Conversation history
User preferences
Workflow state
Long-running tasks

Short-Term Memory

Short-term memory stores:

Current conversation context
Immediate workflow information

Long-Term Memory

Long-term memory stores:

Persistent preferences
Historical interactions
Learned context

Workflow State Management

State management tracks:

Current task progress
Intermediate outputs
Pending actions

Human-in-the-Loop (HITL) Workflows

High-risk workflows may require:

Human approvals
Validation checkpoints
Escalation paths

Approval Gates

Approval gates can prevent:

Unsafe actions
Unauthorized tool usage
Harmful outputs

Safety and Governance

Organizations should enforce:

Tool restrictions
Permission boundaries
Safety filters
Approval workflows

Autonomous vs Semi-Autonomous Agents

Autonomous Agents

Can:

Make decisions independently
Execute workflows automatically

Semi-Autonomous Agents

Require:

Human review
Approval checkpoints

Workflow Monitoring

Organizations should monitor:

Tool usage
Failures
Safety violations
Latency
Costs

Trace Logging

Trace logging helps track:

Workflow execution
Tool calls
Reasoning steps
Agent decisions

Error Handling in Workflows

Workflow pipelines should handle:

API failures
Missing data
Timeout errors
Invalid outputs

Retry Strategies

Common retry strategies include:

Automatic retries
Fallback workflows
Alternative tool selection

Fallback Models

Applications may use fallback models when:

Primary models fail
Costs exceed thresholds
Latency becomes excessive

Workflow Optimization

Optimization strategies include:

Parallel processing
Caching
Smaller models
Efficient retrieval

Latency Considerations

Complex workflows may increase latency due to:

Multiple model calls
Tool invocations
Retrieval operations

Cost Considerations

Tool-augmented systems may increase:

Token usage
API calls
Infrastructure costs

Azure AI Foundry Workflow Capabilities

Azure AI Foundry supports:

Model orchestration
Tool integration
Agent workflows
Evaluation pipelines
Monitoring

Common AI-103 Workflow Scenarios

Scenario 1: Enterprise Research Assistant

Requirements:

Multi-document retrieval
Summarization
Citation generation

Recommended Workflow:

RAG + multistep reasoning

Scenario 2: Customer Service Agent

Requirements:

CRM access
Ticket management
Escalation workflows

Recommended Workflow:

Tool-augmented agent

Scenario 3: Financial Approval System

Requirements:

Risk evaluation
Human approvals
Audit logging

Recommended Workflow:

HITL approval pipeline

Scenario 4: AI Coding Assistant

Requirements:

Code generation
Code execution
Documentation retrieval

Recommended Workflow:

Code model + tool orchestration

Common AI-103 Exam Tips

Understand Workflow Patterns

Know:

Sequential workflows
Parallel workflows
Conditional workflows

Learn Tool-Augmented AI Concepts

Understand:

Function calling
Tool orchestration
Dynamic tool selection

Understand Multistep Reasoning

Know:

Chain-of-thought reasoning
Planner-executor patterns
ReAct workflows

Learn Governance Concepts

Understand:

HITL workflows
Approval gates
Monitoring
Trace logging

Summary

Modern AI applications increasingly rely on:

Workflow orchestration
Tool augmentation
Multistep reasoning
Agentic architectures

For the AI-103 exam, you should understand:

AI workflow design
Function calling
Tool orchestration
Sequential and parallel workflows
Multistep reasoning
Planner-executor architectures
ReAct patterns
Memory integration
HITL workflows
Monitoring and governance

These concepts enable organizations to build:

Intelligent
Autonomous
Scalable
Governed AI systems

They are foundational for modern generative AI and agentic solutions on Azure.

Practice Exam Questions

Question 1

What is the primary purpose of tool augmentation in AI systems?

A. Reduce storage costs
B. Extend model capabilities using external tools
C. Eliminate prompts
D. Replace vector search

Answer

B. Extend model capabilities using external tools

Explanation

Tool augmentation enables AI systems to interact with APIs, databases, and other services.

Question 2

What does function calling enable a model to do?

A. Generate only static responses
B. Invoke external tools using structured inputs
C. Eliminate workflows
D. Replace embeddings

Answer

B. Invoke external tools using structured inputs

Explanation

Function calling allows models to interact with external services.

Question 3

Which workflow type executes tasks simultaneously?

A. Sequential workflow
B. Parallel workflow
C. Manual workflow
D. Static workflow

Answer

B. Parallel workflow

Explanation

Parallel workflows improve speed by running tasks concurrently.

Question 4

What is multistep reasoning?

A. Compressing vector indexes
B. Breaking complex tasks into smaller reasoning steps
C. Increasing GPU memory
D. Reducing prompt size only

Answer

B. Breaking complex tasks into smaller reasoning steps

Explanation

Multistep reasoning improves problem-solving accuracy.

Question 5

What does the ReAct pattern combine?

A. Compression and storage
B. Reasoning and acting
C. Replication and scaling
D. Encryption and backup

Answer

B. Reasoning and acting

Explanation

ReAct combines reasoning steps with tool usage.

Question 6

What is the purpose of workflow state management?

A. Monitor GPU temperature
B. Track task progress and intermediate outputs
C. Disable logging
D. Replace semantic search

Answer

B. Track task progress and intermediate outputs

Explanation

State management helps maintain workflow continuity.

Question 7

Which architecture separates planning from execution?

A. Static inference architecture
B. Planner-executor architecture
C. Batch storage architecture
D. Compression architecture

Answer

B. Planner-executor architecture

Explanation

Planner-executor systems divide reasoning and execution responsibilities.

Question 8

Why are approval gates important in AI workflows?

A. They increase vector dimensions
B. They prevent unsafe or unauthorized actions
C. They reduce indexing speed
D. They eliminate monitoring requirements

Answer

B. They prevent unsafe or unauthorized actions

Explanation

Approval gates enforce governance and human oversight.

Question 9

Which concept allows AI systems to remember previous interactions?

A. Semantic ranking
B. Memory integration
C. Static chunking
D. GPU partitioning

Answer

B. Memory integration

Explanation

Memory enables contextual continuity and long-running workflows.

Question 10

What is a major challenge of complex AI workflows?

A. Eliminating all costs
B. Increased latency from multiple operations
C. Removing all need for monitoring
D. Preventing all hallucinations automatically