This post is a part of the AI-103: Develop AI Apps and Agents on Azure Exam Prep Hub. 
This topic falls under these sections:
Plan and manage an Azure AI solution (25–30%)
   --> Set up AI solutions in Foundry
      --> Integrate Foundry projects with Continuous Integration and Continuous Deployment (CI/CD) pipelines

Note that there are 10 practice questions (with answers and explanations) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern AI applications and agent-based systems are continuously evolving.

Organizations frequently update:

AI models
Prompts
Agent workflows
APIs
Retrieval systems
Infrastructure
Security configurations

Manual deployment processes are slow, error-prone, and difficult to scale.

To solve these challenges, organizations use:

Continuous Integration (CI)
Continuous Deployment (CD)
Automated testing
Infrastructure-as-Code (IaC)
Automated validation pipelines

The AI-103: Develop AI Apps and Agents on Azure certification exam tests your understanding of how to integrate Azure AI Foundry projects into CI/CD pipelines.

For the AI-103 exam, you should understand:

CI/CD concepts
Azure DevOps pipelines
GitHub Actions workflows
Infrastructure-as-Code
Automated AI deployment workflows
Model versioning
Deployment automation
Testing and validation
Environment management
Rollback strategies
Monitoring deployment health

What Is CI/CD?

CI/CD stands for:

Continuous Integration
Continuous Deployment (or Continuous Delivery)

CI/CD automates software and AI deployment processes.

Continuous Integration (CI)

Continuous Integration focuses on:

Automatically building code
Running automated tests
Validating changes
Detecting issues early

Developers frequently merge changes into shared repositories.

Continuous Deployment (CD)

Continuous Deployment automates:

Application releases
Model deployments
Infrastructure updates
Environment promotion

CD ensures new versions are deployed safely and consistently.

Why CI/CD Matters for AI Solutions

AI systems are more complex than traditional applications because they include:

Models
Prompts
Retrieval pipelines
Vector indexes
Agent workflows
Tool integrations

CI/CD helps ensure:

Reliable deployments
Repeatable processes
Faster releases
Reduced downtime
Safer experimentation

Azure AI Foundry and CI/CD

Azure AI Foundry integrates with:

Azure DevOps
GitHub Actions
Infrastructure-as-Code tools
Azure CLI
SDKs
REST APIs

This enables automated AI workflows.

Source Control for AI Projects

AI projects should use source control systems.

Common repositories include:

GitHub
Azure Repos

What Should Be Stored in Source Control?

Common AI assets include:

Application code
Prompt templates
Agent configurations
Infrastructure definitions
Deployment scripts
Evaluation workflows
Test cases
CI/CD pipeline definitions

What Should NOT Be Stored in Source Control?

Never store:

Secrets
API keys
Passwords
Certificates
Sensitive credentials

Use Azure Key Vault instead.

Azure DevOps

Azure DevOps provides:

Repositories
Build pipelines
Release pipelines
Work tracking
Artifact management

Azure DevOps is commonly used for enterprise AI deployments.

GitHub Actions

GitHub Actions supports:

Automated workflows
Build automation
Testing pipelines
Deployment automation
CI/CD orchestration

GitHub Actions is widely used for AI applications hosted in GitHub repositories.

Infrastructure-as-Code (IaC)

Infrastructure-as-Code automates infrastructure provisioning.

Instead of manually creating resources, infrastructure is defined in code.

Benefits of IaC

IaC provides:

Repeatability
Version control
Consistency
Automation
Reduced configuration drift

Common IaC Tools in Azure

Common Azure IaC tools include:

ARM templates
Bicep
Terraform

Bicep

Bicep is a declarative language for Azure infrastructure.

Used to deploy:

Azure OpenAI resources
Azure AI Search
Storage accounts
Networking resources
Key Vault
App Services

Terraform

Terraform is a multi-cloud Infrastructure-as-Code tool.

Useful for:

Hybrid environments
Multi-cloud deployments
Large enterprise automation

Automating Azure AI Resource Deployment

CI/CD pipelines can automatically provision:

Azure OpenAI
Azure AI Search
Cosmos DB
Azure Functions
App Service
Networking
Monitoring services

Automating Model Deployments

Model deployment pipelines may automate:

Model version selection
Deployment creation
Endpoint configuration
Scaling configuration
Rollback management

Model Versioning

Versioning is critical for AI deployments.

Benefits include:

Safer updates
Rollback support
Testing new versions
Comparing performance

Environment Management

AI solutions commonly use multiple environments.

Typical environments include:

Development
Testing
Staging
Production

Development Environment

Used for:

Experimentation
Initial testing
Prompt development
Rapid iteration

Testing Environment

Used for:

Automated testing
Integration testing
Validation workflows

Staging Environment

Used for:

Final validation
Production-like testing
User acceptance testing

Production Environment

Used for:

Live workloads
Enterprise applications
Customer-facing systems

Production environments require:

Strong monitoring
Security controls
Scalability
High availability

Automated Testing in AI Pipelines

Testing AI systems is more complex than traditional software testing.

AI pipelines should validate:

Functional behavior
Prompt quality
Retrieval quality
Latency
Safety
Reliability

Unit Testing

Unit testing validates:

Individual functions
APIs
Tool integrations
Components

Integration Testing

Integration testing validates interactions between:

Models
APIs
Search systems
Databases
Agents

Prompt Evaluation

Prompt evaluation helps assess:

Response quality
Groundedness
Hallucinations
Relevance
Consistency

Automated Evaluation Pipelines

Evaluation pipelines may measure:

Accuracy
Latency
Token usage
Toxicity
Retrieval precision

Prompt Flow and CI/CD

Prompt Flow can integrate into CI/CD pipelines.

Prompt Flow supports:

Workflow orchestration
Evaluation pipelines
Prompt testing
Tool integration

Deployment Strategies

Safe deployment strategies reduce risk.

Blue-Green Deployments

Blue-green deployments use two environments:

Current production environment
New deployment environment

Traffic switches after validation.

Benefits:

Reduced downtime
Easy rollback
Safer deployments

Canary Deployments

Canary deployments release updates gradually.

Benefits:

Reduced deployment risk
Easier issue detection
Controlled rollout

Rolling Deployments

Rolling deployments update systems incrementally.

Benefits:

Minimal downtime
Gradual infrastructure replacement

Rollback Strategies

Rollback mechanisms are critical.

Rollbacks may restore:

Previous model versions
Prior prompts
Earlier infrastructure states

Deployment Approval Gates

Approval gates help control production releases.

Approvals may be required before:

Production deployment
Model upgrades
Infrastructure changes

Security in CI/CD Pipelines

Security is a major AI-103 topic.

Azure Key Vault Integration

Pipelines should retrieve secrets securely from:

Azure Key Vault

Examples include:

API keys
Connection strings
Certificates

Managed Identities

Managed identities reduce the need for stored credentials.

Benefits:

Improved security
Simplified authentication
Reduced secret exposure

Role-Based Access Control (RBAC)

RBAC limits access to:

Deployments
Resources
Pipelines
Secrets

Monitoring CI/CD Pipelines

Pipelines should monitor:

Build failures
Deployment failures
Performance regressions
AI quality degradation

Azure Monitor

Azure Monitor supports:

Metrics
Alerts
Logging
Diagnostics

Application Insights

Application Insights helps monitor:

API latency
Failures
Dependency performance
User behavior

AI-Specific Monitoring

AI systems should monitor:

Token usage
Hallucination rates
Retrieval quality
Tool execution failures
Prompt performance

Common AI-103 CI/CD Scenarios

Scenario 1: Enterprise AI Copilot

Requirements:

Frequent prompt updates
Safe production releases
Automated testing

Recommended Approach:

GitHub Actions
Prompt Flow evaluations
Canary deployments

Scenario 2: Large-Scale AI Platform

Requirements:

Infrastructure automation
Multi-environment deployment
Enterprise governance

Recommended Approach:

Azure DevOps
Bicep or Terraform
Approval gates

Scenario 3: AI Agent Workflow System

Requirements:

Frequent workflow updates
Tool integration testing
Prompt validation

Recommended Approach:

Automated evaluation pipelines
Integration testing
Blue-green deployment strategy

Cost Optimization in CI/CD

CI/CD pipelines can increase operational costs.

Cost Optimization Strategies

Use Automated Cleanup

Remove:

Temporary environments
Test resources
Unused deployments

Optimize Test Frequency

Run expensive evaluations only when necessary.

Use Smaller Models for Testing

Smaller models reduce:

Token usage
Compute costs
Evaluation expenses

Common AI-103 Exam Tips

Understand CI/CD Fundamentals

Know:

Continuous Integration
Continuous Deployment
Automated testing
Deployment automation

Learn Deployment Strategies

Understand:

Blue-green deployments
Canary deployments
Rolling deployments
Rollback strategies

Know Infrastructure-as-Code Concepts

Understand:

Bicep
Terraform
ARM templates

Understand AI-Specific Testing

AI systems require testing for:

Prompt quality
Groundedness
Safety
Retrieval accuracy
Latency

Summary

Integrating Azure AI Foundry projects with CI/CD pipelines enables organizations to:

Automate deployments
Improve reliability
Increase scalability
Reduce operational risk
Accelerate AI delivery

For the AI-103 exam, you should understand:

CI/CD fundamentals
Azure DevOps pipelines
GitHub Actions workflows
Infrastructure-as-Code
Automated AI deployment strategies
Environment management
AI testing pipelines
Monitoring and observability
Secure deployment practices
Rollback and release strategies

Strong CI/CD practices are essential for building production-grade AI applications and agent-based systems on Azure.

Managed identities provide secure authentication without storing credentials.

Go to the AI-103 Exam Prep Hub main page