This post is a part of the AI-200: Developing AI Cloud Solutions on Azure  Exam Prep Hub.
This topic falls under these sections:
Secure, monitor, and troubleshoot Azure solutions (20–25%)
   --> Monitor and troubleshoot Azure solutions
      --> Trace distributed systems by using OpenTelemetry SDKs

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 4 practice tests with 30 questions each available from the hub's main page below the exam topics section.

Overview

Modern AI applications are rarely single-process applications. A typical solution might include an API hosted in Azure App Service or Azure Container Apps, Azure Functions for background processing, Azure Service Bus or Event Grid for messaging, a database such as Azure Cosmos DB or Azure Database for PostgreSQL, and one or more AI services.

When a request travels through several of these components, determining where time was spent, where an error occurred, or which downstream dependency caused a failure can be difficult if each component produces isolated logs.

OpenTelemetry (OTel) addresses this problem by providing a vendor-neutral framework for generating, collecting, and exporting telemetry—including traces, metrics, and logs. For AI-200, an especially important capability is distributed tracing, which allows a request to be followed across application and service boundaries.

The key exam skill is understanding how OpenTelemetry SDKs create spans, associate spans into traces, propagate trace context between services, and export telemetry to an observability backend.

1. What Is Distributed Tracing?

Distributed tracing tracks a single logical operation as it moves through multiple services, processes, and infrastructure components.

Consider an AI application with this architecture:

Client
   │
   ▼
API
   │
   ├──► Azure Cosmos DB
   │
   ├──► Azure OpenAI
   │
   └──► Azure Service Bus
             │
             ▼
       Azure Function
             │
             ▼
       PostgreSQL

A user might submit a question to the API. The API retrieves information from Cosmos DB, calls an AI model, places a message on Service Bus, and an Azure Function processes the message.

Without distributed tracing, each component might generate its own logs:

API log:
Request completed in 2.8 seconds
Cosmos DB log:
Query completed in 150 ms
Azure Function log:
Execution completed in 1.9 seconds

It can be difficult to determine whether these records belong to the same user request.

With distributed tracing, OpenTelemetry can associate the operations with a common Trace ID:

Trace ID: 7bba9f...
└── API request
    ├── Cosmos DB query
    ├── Azure OpenAI request
    └── Service Bus operation
         └── Function execution
              └── PostgreSQL query

This allows developers to visualize the complete path of a request and identify slow or failing components.

A trace is composed of spans, with each span representing an individual operation. Spans can be nested to represent parent-child relationships.

2. Trace vs. Span

These two terms are fundamental to the AI-200 topic.

Trace

A trace represents the complete journey of a logical operation through a distributed system.

For example:

Trace
│
├── HTTP request
│
├── Database query
│
├── AI model request
│
└── Message processing

A trace is identified by a Trace ID.

Span

A span represents a single unit of work within the trace.

Examples include:

• An HTTP request
• A database query
• An Azure SDK operation
• An RPC call
• A call to an AI service
• A message-processing operation
• A custom application operation

A span typically contains information such as:

• Span name
• Trace ID
• Span ID
• Parent span ID
• Start time
• End time
• Attributes
• Events
• Status
• Links

For example:

Trace ID: ABC123
Span: HTTP GET /orders
    │
    ├── Span: SQL SELECT
    │
    └── Span: HTTP GET /customer

The parent-child relationship allows the tracing system to reconstruct the request’s execution path.

3. The OpenTelemetry API and SDK

OpenTelemetry separates the API from the SDK.

OpenTelemetry API

The API provides interfaces that application code and instrumentation can use to create telemetry.

For tracing, the API includes concepts such as:

• TracerProvider
• Tracer
• Span
• SpanContext

OpenTelemetry SDK

The SDK provides the implementation responsible for processing and exporting telemetry.

The SDK can handle:

• Span creation
• Sampling
• Span processing
• Exporting
• Resource information
• Propagation configuration

A TracerProvider is generally initialized as part of application startup and is used to create Tracer instances.

Conceptually:

Application
    │
    ▼
TracerProvider
    │
    ▼
Tracer
    │
    ▼
Span
    │
    ▼
Span Processor
    │
    ▼
Exporter
    │
    ▼
Telemetry backend

4. What Is a Tracer?

A Tracer creates spans.

For example, an application might obtain a tracer for its order-processing component:

Tracer
   │
   ├── Span: Validate order
   ├── Span: Retrieve customer
   └── Span: Submit payment

The tracer itself does not represent the operation. Instead, it is the mechanism used to create spans describing operations.

A common pattern is to initialize the tracing infrastructure once and then obtain tracers from the configured TracerProvider.

5. Span Context

A SpanContext contains the information necessary to identify and propagate a span’s tracing context.

Important fields include:

• Trace ID — identifies the overall trace.
• Span ID — identifies the current span.
• Trace flags — include information such as whether the trace is sampled.
• Trace state — can carry tracing-system-specific information.

The SpanContext is especially important because it is the portion of tracing information that can be serialized and propagated between processes.

For example:

Service A
Trace ID = 123
Span ID  = ABC
        │
        │ propagate context
        ▼
Service B
Trace ID = 123
Span ID  = XYZ
Parent   = ABC

Service B creates a new span but associates it with the existing trace.

6. Context Propagation

Context propagation is the key concept behind distributed tracing.

Suppose Service A calls Service B:

Service A
   │
   │ HTTP request
   ▼
Service B

Service A needs to transmit its tracing context with the request.

Service B then extracts that context and creates a child span.

Service A
Trace ID = 123
Span ID  = AAA
      │
      │ trace context
      ▼
Service B
Trace ID = 123
Span ID  = BBB
Parent   = AAA

The result is a single trace containing both operations.

OpenTelemetry commonly uses the W3C Trace Context format for this purpose. HTTP requests can carry trace context using headers such as traceparent.

Why this matters

Without context propagation:

Service A → Trace A
Service B → Trace B

The observability platform cannot reliably determine that the operations belong to the same request.

With context propagation:

Service A ───────┐
                 │
                 ▼
              Trace 123
                 ▲
                 │
Service B ───────┘

The complete distributed operation can be reconstructed.

7. Automatic vs. Manual Context Propagation

In many applications, instrumentation libraries automatically inject and extract trace context.

For example:

HTTP client
    │
    ▼
Instrumentation
    │
    ├── inject trace context
    ▼
HTTP request

The receiving service’s instrumentation can extract the context automatically.

This is preferred because it reduces custom tracing code and helps maintain consistent propagation behavior. OpenTelemetry documentation notes that instrumentation libraries handle propagation automatically for many common scenarios.

Manual propagation may be necessary when:

• A custom transport is being used.
• A messaging protocol is not automatically instrumented.
• Application-specific integration is required.
• The developer needs explicit control over propagation.

The general concepts are:

Inject

Current Context
       │
       ▼
Propagator
       │
       ▼
Outgoing message

Extract

Incoming message
       │
       ▼
Propagator
       │
       ▼
Remote Context

The OpenTelemetry Propagators API provides mechanisms for injecting and extracting context from messages.

8. Distributed Tracing Across Messaging Systems

Distributed systems don’t communicate only through HTTP.

AI applications frequently use:

• Azure Service Bus
• Azure Event Grid
• Queues
• Event streams
• Background workers

For example:

API
 │
 │ send message
 ▼
Service Bus
 │
 │ receive message
 ▼
Azure Function

The original request may create one trace, while the message-processing operation occurs later and potentially on another compute instance.

Tracing context can be propagated through messaging metadata when supported and correctly configured.

This allows developers to understand relationships such as:

Trace
│
├── API request
│
└── Message publishing
       │
       └── Message processing
              │
              └── Database operation

An important distinction is that asynchronous processing can have different causal relationships from a simple synchronous HTTP call. OpenTelemetry supports Span Links for situations where an operation is related to another span but doesn’t necessarily fit a straightforward parent-child hierarchy.

9. Span Attributes

Attributes are key-value pairs attached to spans.

They provide additional information about an operation.

For example:

Span:
    Name: GET /orders
Attributes:
    http.request.method = GET
    http.route = /orders
    customer.tier = premium
    order.type = subscription

Attributes can help developers filter and analyze telemetry.

However, developers should avoid placing sensitive information into telemetry.

For example, avoid attributes containing:

• Passwords
• Access keys
• Authentication tokens
• Credit-card information
• Sensitive personal information

The same caution applies to OpenTelemetry Baggage, because baggage can be propagated between services. OpenTelemetry specifically recommends avoiding sensitive data in baggage.

10. Span Events

A span can contain events representing notable occurrences during an operation.

For example:

Span: ProcessOrder
Events:
    10:01:02 - ValidationStarted
    10:01:03 - ValidationCompleted
    10:01:04 - PaymentSubmitted

Events are useful when a developer needs more detail about what happened during a span without creating a separate span for every small occurrence.

11. Span Status

A span can have a status indicating the outcome of an operation.

For example:

Status: OK

or:

Status: ERROR

An error status can help identify failed operations when examining distributed traces.

For example:

Trace
│
├── API request          OK
│
├── Cosmos DB query      OK
│
└── AI service request   ERROR

This immediately focuses troubleshooting on the AI service operation.

12. Resources

OpenTelemetry also associates telemetry with resources.

A resource describes the entity producing the telemetry.

Examples include:

• Service name
• Service version
• Host
• Container
• Kubernetes pod
• Kubernetes namespace
• Cloud environment

For example:

Service:
    order-api
Version:
    2.4.0
Environment:
    production
Container:
    order-api-7d9f

Resource information becomes particularly useful when many instances of the same application generate telemetry.

OpenTelemetry defines resources as information describing the entity for which telemetry is recorded.

13. Exporters

Creating spans is only part of the process. The telemetry needs to be sent somewhere where it can be analyzed.

An exporter sends telemetry to a destination.

Conceptually:

Application
    │
    ▼
OpenTelemetry SDK
    │
    ▼
Span Processor
    │
    ▼
Exporter
    │
    ▼
Observability backend

Possible destinations include:

• OpenTelemetry Collector
• Azure Monitor
• Other observability platforms
• Console output for development/testing

OpenTelemetry is vendor-neutral, so applications can use exporters appropriate to their target telemetry backend.

14. OpenTelemetry Collector

The OpenTelemetry Collector provides a vendor-neutral way to receive, process, and export telemetry.

A common architecture is:

Application A ─┐
Application B ─┼──► OpenTelemetry Collector ───► Backend
Application C ─┘

The Collector can act as an intermediary between applications and observability platforms.

This can be valuable when an organization wants to:

• Centralize telemetry processing
• Change telemetry destinations without modifying every application
• Filter or transform telemetry
• Batch telemetry
• Route telemetry to different destinations

The Collector is separate from the OpenTelemetry SDK running inside the application.

15. Sampling

Large distributed applications can generate enormous numbers of spans.

Sampling controls how much tracing data is collected.

For example, an application processing one million requests per day may not need to retain every successful request.

A sampling strategy might retain:

• 100% of errors
• 100% of slow requests
• A percentage of successful requests

Conceptually:

1,000,000 requests
       │
       ▼
   Sampler
       │
       ├── 10% normal requests
       └── 100% important/error requests

Sampling reduces telemetry volume, storage requirements, and processing overhead.

OpenTelemetry supports sampling decisions at different stages of telemetry collection.

Exam point

Do not confuse sampling with filtering at the observability backend.

Sampling can influence whether a span is recorded/exported in the first place, whereas backend filtering occurs after telemetry has already reached the collection pipeline.

16. Span Processors

A SpanProcessor receives spans during their lifecycle and passes them through the telemetry pipeline.

Conceptually:

Span
 │
 ▼
Span Processor
 │
 ▼
Exporter

OpenTelemetry supports processors such as:

• Simple span processing
• Batch span processing

A batch processor can accumulate spans and export them together rather than exporting every span immediately.

This can improve efficiency and reduce the overhead associated with frequent network calls.

17. Instrumentation

Instrumentation is the process of adding telemetry generation to an application.

There are two broad approaches.

Automatic instrumentation

Automatic instrumentation uses libraries, agents, or platform capabilities to instrument common frameworks and dependencies.

Examples may include automatically tracing:

• HTTP requests
• HTTP clients
• Database calls
• Framework operations
• Messaging operations

This is generally the easiest way to get broad application coverage.

Manual instrumentation

Manual instrumentation allows developers to explicitly create spans around application-specific operations.

For example:

Start span: GenerateAnswer
    Retrieve documents
    Build prompt
    Call model
    Process response
End span: GenerateAnswer

Manual instrumentation is particularly useful for business operations that automatic instrumentation doesn’t understand.

18. Custom Spans

Suppose an AI application performs a business operation called GenerateRecommendation.

An HTTP instrumentation library may capture the HTTP request, but that doesn’t necessarily describe the application’s internal business process.

A developer can create a custom span:

Trace
│
└── HTTP POST /recommend
      │
      └── GenerateRecommendation
            │
            ├── RetrieveDocuments
            └── CallAIModel

This provides much better visibility into application-specific processing.

A good custom span should represent a meaningful unit of work—not every individual line of code.

19. Trace Context and the W3C Trace Context Standard

For distributed tracing to work across different technologies, services need a common format for transmitting tracing information.

OpenTelemetry commonly uses the W3C Trace Context specification.

An HTTP request can contain a traceparent header carrying tracing information.

Conceptually:

traceparent:
00-<trace-id>-<parent-span-id>-<flags>

The receiving service extracts the information and uses it when creating its span.

This is one of the most important mechanisms that allows heterogeneous applications to participate in the same distributed trace.

20. OpenTelemetry in Azure Applications

For AI-200, think of OpenTelemetry as a technology that can span the entire Azure application architecture.

For example:

                    ┌───────────────┐
                    │   Client      │
                    └───────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │ API / App     │
                    └───────┬───────┘
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
          Cosmos DB     AI Service    Service Bus
                                          │
                                          ▼
                                   Azure Function
                                          │
                                          ▼
                                     PostgreSQL

A properly instrumented solution can create a trace that makes the entire processing path observable.

This is especially valuable for AI applications because an apparently slow API request may actually be caused by:

• A database query
• A vector search
• An AI model request
• A downstream HTTP service
• A message-processing delay
• A function execution
• A retry
• Network latency

Distributed tracing helps identify which operation actually contributed to the latency.

21. Troubleshooting With Distributed Traces

Consider an API that normally responds in 500 ms but suddenly takes 6 seconds.

A conventional application log might show:

POST /chat completed in 6 seconds

That tells you the symptom but not the cause.

A distributed trace might show:

POST /chat                         6.0 sec
│
├── Authentication                50 ms
├── Cosmos DB query               100 ms
├── Vector search                 250 ms
├── AI model request             5.4 sec
└── Response processing          200 ms

Now the likely problem is immediately visible.

Another trace might show:

POST /chat                         6.0 sec
│
├── Cosmos DB query               100 ms
├── Service Bus send               20 ms
└── Function processing          5.8 sec
      │
      └── PostgreSQL query       5.6 sec

The problem is now much more likely to be the database operation rather than the API itself.

22. Logs, Metrics, and Traces Work Together

OpenTelemetry supports multiple telemetry signals:

For example:

Metric

API latency = 2.4 seconds average

Trace

API
 └── Database
      └── Query = 2.1 seconds

Log

Database timeout threshold exceeded

Together, they provide much more useful observability than any individual signal.

OpenTelemetry is designed to support these telemetry signals and their correlation.

23. Traces vs. Metrics: A Common Exam Distinction

A common exam scenario might say:

“The development team needs to determine which downstream service is responsible for the latency of an individual request.”

The best answer is generally distributed tracing.

If the requirement is:

“Determine the average request latency across all requests during the past hour.”

A metric is generally more appropriate.

If the requirement is:

“Find the detailed error message generated during a particular request.”

A log may provide the necessary detail.

Remember:

Trace  → Where did this request go?
Metric → How is the system behaving overall?
Log    → What specifically happened?

These signals complement rather than replace one another.

24. Best Practices for OpenTelemetry Distributed Tracing

1. Instrument meaningful operations

Create spans around meaningful operations rather than excessive low-level code.

Good:

GenerateEmbedding
QueryVectorDatabase
CallAIModel

Less useful:

AssignVariable
IncrementCounter
EnterIfStatement

2. Use automatic instrumentation where practical

Automatic instrumentation can provide broad coverage with less development effort.

3. Add useful attributes

Capture information that helps diagnose problems, while avoiding secrets and sensitive data.

4. Preserve context across service boundaries

If trace context is lost between services, the distributed trace may become fragmented.

5. Use sampling appropriately

High-volume applications can use sampling to reduce telemetry overhead.

6. Use batching when appropriate

Batch processing can reduce the overhead of exporting individual spans.

7. Give services meaningful resource information

Service names, versions, environments, and deployment information make telemetry easier to interpret.

8. Protect telemetry

Tracing data can contain sensitive information. Do not treat telemetry as automatically safe simply because it is operational data.

25. Important AI-200 Concepts to Remember

For the exam, make sure you can distinguish these concepts:

26. Exam Scenario: Putting It All Together

Imagine an AI chatbot architecture:

User
 │
 ▼
Azure App Service
 │
 ├──► Azure Cosmos DB
 │
 ├──► AI model
 │
 └──► Azure Service Bus
             │
             ▼
       Azure Function
             │
             ▼
       PostgreSQL

The application is instrumented with OpenTelemetry.

A request generates:

Trace ID = 12345
Span 1: HTTP POST /chat
    │
    ├── Span 2: Cosmos DB query
    │
    ├── Span 3: AI model request
    │
    └── Span 4: Service Bus publish
              │
              └── Span 5: Function processing
                        │
                        └── Span 6: PostgreSQL query

The important concepts are:

1. The trace represents the overall operation.
2. Each span represents a unit of work.
3. The Trace ID associates the spans.
4. Context propagation allows tracing information to cross service boundaries.
5. Span attributes provide additional diagnostic information.
6. Span events record significant occurrences.
7. The SDK processes the telemetry.
8. A span processor manages the span processing pipeline.
9. An exporter sends telemetry to a destination.
10. Sampling can reduce telemetry volume.
11. An OpenTelemetry Collector can provide an intermediary telemetry pipeline.

If you understand that flow, you have the foundation needed for most AI-200 questions involving OpenTelemetry.

Practice Exam Questions

Question 1

An AI application consists of an API, an Azure Function, and a database. A developer wants to follow a single user request across all three components.

Which OpenTelemetry capability is most important?

A. Resource tagging
B. Metric aggregation
C. Log rotation
D. Context propagation

Answer: D

Explanation: Context propagation allows tracing information to travel across process and service boundaries. This enables spans generated by different components to be associated with the same trace. Without propagation, each service could create an isolated trace.

Question 2

An application creates a trace for an HTTP request. The request then causes a database query and a call to an AI service.

What should represent the database query and AI service call?

A. Separate resources
B. Separate spans within the trace
C. Separate TraceProviders
D. Separate exporters

Answer: B

Explanation: A span represents a unit of work. The database query and AI service call can each be represented by spans that belong to the overall trace.

Question 3

An organization wants to reduce the amount of tracing data generated by a high-volume application while continuing to collect a representative subset of traces.

Which OpenTelemetry capability should be configured?

A. Propagation
B. Span attributes
C. Resource detection
D. Sampling

Answer: D

Explanation: Sampling controls which traces or spans are recorded and/or exported. It is commonly used to reduce telemetry volume and overhead in high-volume applications.

Question 4

Service A sends an HTTP request to Service B. Service B must create a span that belongs to the same distributed trace as the request from Service A.

What must occur?

A. Service B must use the same Span ID as Service A.
B. Service A and Service B must use the same Tracer instance.
C. Service B must export its telemetry before Service A.
D. Trace context must be propagated from Service A to Service B.

Answer: D

Explanation: Trace context propagation allows Service B to obtain the Trace ID and parent span information from Service A. Service B creates its own span while maintaining the relationship with the existing trace. The child span should have its own Span ID.

Question 5

A developer wants to attach information such as order.type = subscription to a span representing an order-processing operation.

What should the developer use?

A. Span attribute
B. Span exporter
C. Trace ID
D. Propagator

Answer: A

Explanation: Span attributes are key-value pairs used to add metadata to spans. They can make traces easier to filter, search, and analyze.

Question 6

An application uses OpenTelemetry and needs to send collected spans to a telemetry backend.

Which component is responsible for sending the telemetry to the destination?

A. Tracer
B. Exporter
C. SpanContext
D. Resource

Answer: B

Explanation: An exporter sends telemetry to a destination such as an OpenTelemetry Collector, Azure Monitor, or another supported observability backend.

Question 7

An application uses a custom messaging mechanism that isn’t automatically instrumented. The developer needs to transfer OpenTelemetry trace context through the message.

Which OpenTelemetry concept is specifically designed to inject and extract context from messages?

A. Resource
B. Span Event
C. Propagator
D. Sampler

Answer: C

Explanation: Propagators provide mechanisms for injecting context into and extracting context from carriers such as HTTP headers or message metadata.

Question 8

A development team needs to determine which operation caused an individual API request to take 8 seconds. The API calls three downstream services.

Which telemetry signal is most appropriate for following the request through the individual services?

A. Distributed trace
B. Aggregate metric
C. Static configuration
D. Resource definition

Answer: A

Explanation: Distributed tracing is specifically designed to follow individual operations across distributed components. A trace can reveal which downstream operation consumed most of the 8 seconds.

Question 9

An application generates millions of spans. The development team wants to process spans in groups before exporting them to reduce the overhead associated with exporting each span individually.

Which component is relevant to this requirement?

A. Trace ID
B. Batch span processor
C. Propagator
D. SpanContext

Answer: B

Explanation: A batch span processor collects spans and exports them in batches. This can improve efficiency compared with exporting every span individually.

Question 10

An AI application sends trace context to an external service. Developers are considering adding user credentials and other sensitive information to OpenTelemetry baggage so that it can be available to downstream services.

What is the best approach?

A. Add the credentials to baggage because baggage is encrypted by OpenTelemetry.
B. Add credentials only to the Trace ID.
C. Store the credentials in span attributes instead.
D. Do not place credentials or other sensitive information in baggage.

Answer: D

Explanation: Baggage can be propagated across service boundaries, so sensitive information placed in baggage may be transmitted to downstream systems. Credentials, API keys, and other sensitive information should not be placed in baggage.

Quick Review

Before taking the AI-200 exam, make sure you can answer these questions confidently:

• What is a trace? — The complete distributed operation.
• What is a span? — An individual unit of work within a trace.
• What creates spans? — A Tracer.
• What provides tracers? — A TracerProvider.
• What connects spans across services? — Context propagation.
• What carries trace/span identity? — SpanContext.
• What injects and extracts propagation data? — Propagators.
• What adds metadata to spans? — Attributes.
• What records occurrences within a span? — Events.
• What sends telemetry somewhere? — An exporter.
• What can batch spans before export? — A span processor.
• What reduces telemetry volume? — Sampling.
• What describes the telemetry-producing entity? — A resource.
• What can receive, process, and forward telemetry? — The OpenTelemetry Collector.
• What lets you follow a request across distributed services? — Distributed tracing.
• What should never be casually placed in telemetry or baggage? — Secrets and sensitive information.

Go to the AI-200 Exam Prep Hub main page