This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Choose the appropriate Foundry services for generative AI and agents
      --> Choose appropriate memory, tool, and knowledge integration services for agent solutions

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI agents are far more advanced than traditional chatbots.

AI agents can:

Reason through problems
Plan tasks
Access tools
Retrieve knowledge
Maintain conversational memory
Execute workflows
Interact with enterprise systems
Coordinate multi-step operations

The AI-103: Develop AI Apps and Agents on Azure certification exam places significant emphasis on understanding how to design and implement these agent capabilities using Azure AI Foundry and related Azure services.

One of the most important skills tested on the exam is the ability to choose appropriate:

Memory systems
Tool integration services
Knowledge integration services
Retrieval architectures
Agent orchestration tools

For the AI-103 exam, you should understand:

Different types of agent memory
Tool calling and function calling
Retrieval-Augmented Generation (RAG)
Knowledge grounding
Azure AI Search integration
Agent orchestration workflows
External API integration
Vector search and embeddings
Enterprise knowledge integration
Security and governance considerations

What Are AI Agents?

AI agents are AI-powered systems capable of:

Interpreting goals
Planning actions
Using tools
Retrieving information
Maintaining context
Completing tasks autonomously or semi-autonomously

Unlike traditional chatbots, AI agents can:

Interact with APIs
Execute workflows
Use memory
Retrieve enterprise knowledge
Chain actions together
Adapt dynamically to user requests

Components of an AI Agent Architecture

Modern AI agent solutions commonly include:

Large Language Models (LLMs)
Memory systems
Retrieval systems
Knowledge integration
Tool and function calling
Workflow orchestration
Security and governance controls

Azure AI Foundry and Agent Solutions

Azure AI Foundry provides services and tools that help developers:

Build AI agents
Integrate tools
Connect enterprise knowledge
Implement RAG
Orchestrate workflows
Evaluate agent behavior
Monitor AI systems

Core services often include:

Azure OpenAI
Azure AI Search
Prompt Flow
Azure AI Content Safety
Azure Functions
Azure Logic Apps
Azure Cosmos DB
Azure SQL Database

Memory in AI Agents

What Is Agent Memory?

Memory enables AI agents to retain and use information over time.

Memory allows agents to:

Maintain conversational context
Remember user preferences
Track workflow state
Store historical interactions
Support long-running tasks

Without memory, every interaction becomes isolated.

Types of Agent Memory

The AI-103 exam may test multiple memory types.

Short-Term Memory

What Is Short-Term Memory?

Short-term memory stores temporary conversational context.

Examples:

Current chat history
Active task context
Immediate instructions

Characteristics of Short-Term Memory

Session-based
Temporary
Fast access
Often stored in prompts or session state

When to Use Short-Term Memory

Use short-term memory for:

Conversational continuity
Current workflow tracking
Multi-turn conversations

Long-Term Memory

What Is Long-Term Memory?

Long-term memory stores persistent information across sessions.

Examples:

User preferences
Historical interactions
Persistent profiles
Prior decisions

Characteristics of Long-Term Memory

Persistent storage
Cross-session continuity
Larger storage capacity
Supports personalization

Azure Services for Long-Term Memory

Common services include:

Azure Cosmos DB
Azure SQL Database
Azure Storage
Vector databases

When to Use Long-Term Memory

Use long-term memory when:

Personalization is required
User preferences must persist
Historical context matters
Long-running workflows exist

Semantic Memory

What Is Semantic Memory?

Semantic memory stores knowledge in embeddings or vectorized formats.

This enables:

Semantic retrieval
Knowledge recall
Contextual understanding
Similarity matching

Semantic Memory in AI Agents

Semantic memory often uses:

Embedding models
Vector search
Azure AI Search

This allows agents to retrieve relevant information dynamically.

Episodic Memory

What Is Episodic Memory?

Episodic memory stores records of past interactions and events.

Examples:

Past conversations
Completed workflows
User activity history

This helps agents maintain continuity across interactions.

Choosing the Correct Memory Type

Use Short-Term Memory When:

Managing active conversations
Maintaining immediate context
Supporting temporary tasks

Use Long-Term Memory When:

Storing persistent user information
Personalizing experiences
Maintaining history across sessions

Use Semantic Memory When:

Retrieving knowledge semantically
Supporting RAG
Performing contextual retrieval

Use Episodic Memory When:

Tracking prior interactions
Supporting historical continuity

Knowledge Integration

What Is Knowledge Integration?

Knowledge integration connects AI agents to external information sources.

Examples:

Enterprise documents
Databases
Knowledge bases
APIs
Websites
Internal systems

Knowledge integration helps agents:

Provide grounded answers
Access current information
Reduce hallucinations
Support enterprise use cases

Retrieval-Augmented Generation (RAG)

What Is RAG?

RAG combines:

Retrieval systems
Search indexes
Embeddings
LLMs

RAG enables agents to retrieve external information before generating responses.

Azure AI Search for Knowledge Integration

Azure AI Search is a core service for:

Vector search
Semantic search
Hybrid search
Enterprise retrieval
Knowledge grounding

It enables agents to:

Search enterprise documents
Retrieve semantically relevant content
Access indexed knowledge

Hybrid Search

Hybrid search combines:

Keyword search
Semantic ranking
Vector search

Hybrid search is often the preferred approach for enterprise AI agents.

Embeddings and Knowledge Retrieval

Embedding models convert content into vector representations.

Embeddings support:

Semantic similarity
Vector retrieval
Knowledge recall
RAG pipelines

Azure OpenAI embedding models are commonly used.

Knowledge Sources for AI Agents

AI agents may integrate with:

Azure Blob Storage
SharePoint
Databases
REST APIs
Enterprise document repositories
CRM systems
ERP systems

Tool Integration

What Is Tool Integration?

Tool integration enables AI agents to interact with external systems.

Examples include:

APIs
Databases
Email systems
Calendars
Search services
Workflow systems

Tool integration allows agents to perform actions instead of only generating text.

Tool Calling and Function Calling

LLMs can invoke:

Tools
Functions
APIs

Examples:

Retrieve weather data
Send emails
Query databases
Create support tickets
Execute workflows

Azure Services for Tool Integration

Common services include:

Azure Functions
Azure Logic Apps
REST APIs
Azure API Management

Azure Functions

Azure Functions provides serverless compute for:

API integrations
Business logic
Event-driven workflows
Tool execution

AI agents often call Azure Functions to execute tasks.

Azure Logic Apps

Azure Logic Apps supports:

Workflow automation
Enterprise integrations
Connector-based orchestration

Logic Apps are useful when:

Multiple systems must interact
Low-code orchestration is preferred
Enterprise automation is needed

Azure API Management

Azure API Management helps:

Secure APIs
Manage API access
Monitor API usage
Apply governance policies

Useful for enterprise AI agent integrations.

Prompt Flow

Prompt Flow is a Foundry tool for:

Building AI workflows
Orchestrating prompts
Chaining tools
Managing agent pipelines
Evaluating workflows

Prompt Flow is a major AI-103 exam topic.

Multi-Agent Systems

Some AI architectures use multiple specialized agents.

Examples:

Research agent
Scheduling agent
Data retrieval agent
Customer service agent

Multi-agent systems may improve:

Scalability
Specialization
Workflow separation

Orchestration Services

Agent orchestration coordinates:

Memory
Retrieval
Tool execution
Workflow management

Common orchestration tools include:

Prompt Flow
Azure Functions
Logic Apps
Custom orchestration frameworks

Security and Governance

AI agent systems require:

Authentication
Authorization
Data protection
Content filtering
Responsible AI controls

Azure AI Content Safety

Azure AI Content Safety helps:

Detect harmful content
Prevent unsafe outputs
Support responsible AI deployments

Role-Based Access Control (RBAC)

RBAC ensures agents only access authorized resources.

This is especially important for:

Enterprise knowledge systems
Confidential data
Regulated environments

Monitoring and Observability

AI agent systems should monitor:

Tool usage
Latency
Errors
Retrieval quality
Hallucinations
Token usage

Monitoring improves:

Reliability
Performance
Troubleshooting

Common AI-103 Scenarios

Scenario 1: Enterprise Copilot

Requirements:

Access enterprise documents
Remember user preferences
Retrieve current information
Support conversational interactions

Recommended Services:

Azure OpenAI
Azure AI Search
Embedding models
Long-term memory storage

Scenario 2: AI Travel Assistant

Requirements:

Access calendars
Book hotels
Query APIs
Manage workflows

Recommended Services:

Azure OpenAI
Tool/function calling
Azure Functions
Prompt Flow

Scenario 3: Customer Support Agent

Requirements:

Retrieve support documents
Track prior interactions
Escalate tickets

Recommended Services:

Azure AI Search
Episodic memory
Azure Functions
CRM integration

Scenario 4: Personalized Learning Assistant

Requirements:

Remember learning preferences
Track progress
Recommend materials

Recommended Services:

Long-term memory
Semantic retrieval
Azure Cosmos DB

Common AI-103 Exam Tips

Understand Memory Types

Know the differences between:

Short-term memory
Long-term memory
Semantic memory
Episodic memory

Know When to Use RAG

Use RAG when:

External knowledge is required
Current data is needed
Hallucination reduction matters

Learn Tool Calling Concepts

Agents use:

Function calling
APIs
Workflows
Tool orchestration

This is commonly tested.

Understand Azure Service Roles

Azure AI Search

Used for:

Retrieval
Vector search
Grounding

Azure Functions

Used for:

Executing logic
Tool integration

Prompt Flow

Used for:

Workflow orchestration
Agent pipelines

Azure Cosmos DB

Used for:

Persistent memory
Long-term storage

Summary

AI agents require more than just language models.

Successful agent solutions combine:

Memory systems
Retrieval systems
Knowledge grounding
Tool integration
Workflow orchestration
Security controls

For the AI-103 exam, you should understand:

Different memory architectures
Tool and function calling
RAG workflows
Azure AI Search integration
Knowledge retrieval strategies
Prompt Flow orchestration
Persistent memory services
Enterprise AI integration patterns

Understanding how these services work together is critical for building scalable and intelligent AI agent solutions.

Azure Cosmos DB is commonly used for scalable persistent memory storage.

Go to the AI-103 Exam Prep Hub main page