DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Process data by using KQL (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform streaming data
      --> Process data by using KQL

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

As organizations increasingly rely on real-time analytics, the ability to query, transform, and analyze streaming data efficiently has become a critical skill for data engineers. Within Microsoft Fabric, one of the most important technologies for real-time data processing is Kusto Query Language (KQL).

KQL is the primary query language used in Real-Time Intelligence, Eventhouses, KQL Databases, and many streaming analytics scenarios within Microsoft Fabric. It is specifically optimized for high-performance analysis of large volumes of telemetry, log, event, and time-series data.

For the DP-700 exam, candidates should understand how KQL is used to process streaming data, when it should be selected over Spark or SQL, common KQL operators, ingestion concepts, aggregation techniques, windowing functions, and real-time analytics patterns.

What Is KQL?

Kusto Query Language (KQL) is a read-optimized query language developed by Microsoft for exploring, analyzing, and transforming large volumes of structured, semi-structured, and streaming data.

KQL is the primary language used in:

Microsoft Fabric Real-Time Intelligence
Eventhouses
KQL Databases
Azure Data Explorer
Microsoft Sentinel
Azure Monitor Logs

KQL is designed for:

Fast interactive analytics
Log analysis
Telemetry processing
Streaming data analytics
Time-series analysis
Monitoring solutions

Unlike traditional T-SQL, KQL uses a pipeline-style syntax that makes analytical queries easier to read and maintain.

Why Use KQL for Streaming Data?

KQL is optimized for scenarios involving:

High ingestion rates
Near real-time querying
Large event volumes
Time-series analysis
Operational monitoring
IoT telemetry
Application logs
Security analytics

A major advantage is that newly ingested streaming data can often be queried within seconds of arrival.

KQL in Microsoft Fabric

Within Microsoft Fabric, KQL is primarily used in:

Eventhouses

Eventhouses provide scalable storage and analytics for real-time data.

Capabilities include:

High-speed ingestion
KQL querying
Streaming analytics
Time-series analysis
Dashboard integration

Eventhouses are commonly used as the central repository for streaming event data.

KQL Databases

A KQL Database is a database inside an Eventhouse.

It stores:

Tables
Functions
Materialized views
Policies

KQL queries execute against these databases.

KQL Processing Workflow

A typical streaming architecture looks like:

			
Event Source
      |
      v
Eventstream
      |
      v
Eventhouse
      |
      v
KQL Database
      |
      v
KQL Queries
      |
      v
Reports / Dashboards

		

Data arrives continuously and becomes available for KQL analysis almost immediately.

Understanding KQL Query Structure

A basic KQL query:

			
Sales
| where Region == "East"
| summarize TotalSales = sum(Amount)

The pipe symbol (|) passes results from one operation to the next.

This pipeline approach is a key exam topic.

Filtering Streaming Data

The where operator filters records.

Example:

			
DeviceReadings
| where Temperature > 100

Common uses:

Error events
High temperatures
Security incidents
Suspicious transactions

Filtering early in a query improves performance.

Selecting Columns

The project operator selects specific columns.

Example:

			
Orders
| project OrderID, CustomerID, Amount

Benefits:

Reduced memory usage
Faster query execution
Cleaner output

Sorting Results

The sort operator orders data.

Example:

			
Orders
| sort by OrderDate desc

This is frequently used in monitoring and dashboard scenarios.

Aggregating Data with Summarize

The summarize operator is one of the most important KQL operators.

Example:

			
Sales
| summarize TotalSales = sum(Amount)

Common aggregation functions:

Function	Purpose
sum()	Total values
avg()	Average
count()	Row count
min()	Minimum value
max()	Maximum value
dcount()	Distinct count

Grouping Data

Grouping is accomplished with summarize and a grouping column.

Example:

			
Sales
| summarize TotalSales=sum(Amount)
    by Region

Output:

Region	TotalSales
East	250000
West	300000

This pattern is heavily used in analytics solutions.

Time-Based Analysis

Streaming data is frequently analyzed by time.

Example:

			
Events
| summarize Count=count()
    by bin(Timestamp, 1h)

The bin() function groups records into fixed time windows.

Common windows:

1 minute
5 minutes
15 minutes
1 hour
1 day

Working with Time-Series Data

Time-series analysis is one of KQL’s strengths.

Example:

			
SensorData
| summarize AvgTemp=avg(Temperature)
    by bin(Timestamp, 5m)

This creates temperature averages every five minutes.

Typical use cases:

IoT monitoring
Server performance
Manufacturing systems
Financial transactions

Parsing Semi-Structured Data

Streaming data often arrives as JSON.

Example:

			
Events
| extend DeviceID = tostring(Event.DeviceID)

Common functions:

Function	Purpose
tostring()	Convert to string
toint()	Convert to integer
todouble()	Convert to decimal
parse_json()	Parse JSON object

Creating Calculated Columns

The extend operator adds calculated values.

Example:

			
Sales
| extend Tax = Amount * .07

Common uses:

Calculations
Data enrichment
Derived metrics

Joining Streaming Data

KQL supports joins between datasets.

Example:

			
Orders
| join Customers
on CustomerID

Common scenarios:

Customer enrichment
Product lookups
Reference data joins

However, excessive joins can impact performance on very large streaming datasets.

Materialized Views

Materialized views precompute query results.

Benefits include:

Faster analytics
Reduced query costs
Improved dashboard performance

Example scenario:

A dashboard continuously displays hourly sales totals.

Instead of recalculating every query, a materialized view stores precomputed results.

This is a frequently tested DP-700 optimization topic.

Update Policies

Update policies automatically transform data during ingestion.

Example:

			
RawEvents Table
        |
Update Policy
        |
ProcessedEvents Table

		

Benefits:

Automatic transformation
Consistent processing
Reduced query complexity

Common use cases:

JSON parsing
Data enrichment
Data normalization

Streaming Ingestion

Fabric supports streaming ingestion into Eventhouses.

Characteristics:

Low latency
High throughput
Near real-time availability

Common sources include:

Eventstreams
Azure Event Hubs
IoT devices
Application telemetry
Custom applications

KQL vs Spark Structured Streaming

DP-700 commonly tests when to choose each technology.

Requirement	KQL	Spark Structured Streaming
Real-time analytics	Excellent	Good
Data science workloads	Limited	Excellent
Machine learning	Limited	Excellent
Interactive querying	Excellent	Moderate
Time-series analysis	Excellent	Good
Large-scale transformations	Moderate	Excellent
SQL-like querying	Excellent	Moderate

Use KQL When:

Analyzing event data
Monitoring telemetry
Building operational dashboards
Performing log analytics
Working with Eventhouses

Use Spark When:

Complex transformations are required
Machine learning workloads exist
Advanced ETL processing is needed
Large-scale data engineering pipelines are required

KQL vs T-SQL

Feature	KQL	T-SQL
Streaming analytics	Excellent	Limited
Time-series analysis	Excellent	Moderate
OLTP operations	Poor	Excellent
Real-time dashboards	Excellent	Moderate
Log analytics	Excellent	Poor

For streaming analytics scenarios in Fabric, KQL is often the preferred option.

Performance Best Practices

Filter Early

Good:

			
Events
| where EventType == "Error"
| summarize count()

Poor:

			
Events
| summarize count()
| where EventType == "Error"

Filtering early reduces processing volume.

Project Only Required Columns

Avoid retrieving unnecessary data.

			
Events
| project Timestamp, DeviceID

Use Materialized Views

For frequently executed analytical queries, materialized views improve performance significantly.

Use Appropriate Time Bins

Choose bin sizes carefully:

Smaller bins = more detailed analysis
Larger bins = better performance

Common DP-700 Exam Scenarios

Scenario 1

You need near real-time analysis of millions of IoT events.

Best choice: Eventhouse + KQL

Scenario 2

You need complex machine learning transformations on streaming data.

Best choice: Spark Structured Streaming

Scenario 3

You need a dashboard showing rolling hourly transaction counts.

Best choice: KQL summarize with bin() function

Scenario 4

You need automatic transformation of incoming JSON data.

Best choice: Update policies

DP-700 Exam Tips

Remember these key points:

KQL is optimized for real-time analytics and event data.
Eventhouses are the primary storage and analytics engine for KQL workloads.
KQL uses a pipeline syntax (|).
where filters data.
project selects columns.
extend creates calculated columns.
summarize performs aggregations.
bin() groups time-series data into intervals.
Materialized views improve query performance.
Update policies automate ingestion-time transformations.
KQL is generally preferred over Spark for interactive streaming analytics.

Practice Exam Questions

Question 1

You need to analyze streaming telemetry data arriving from thousands of IoT devices and provide near real-time dashboards. Which technology should you primarily use?

A. Warehouse stored procedures
B. Dataflow Gen2
C. KQL in an Eventhouse
D. Power Query

Correct Answer: C

Explanation: KQL and Eventhouses are optimized for real-time analytics, telemetry processing, and interactive querying of streaming data.

Question 2

Which KQL operator is used to filter rows from a dataset?

A. summarize
B. where
C. project
D. extend

Correct Answer: B

Explanation: The where operator filters records based on specified conditions.

Question 3

A query needs to calculate total sales by region. Which KQL operator should be used?

A. project
B. where
C. summarize
D. extend

Correct Answer: C

Explanation: summarize performs aggregations such as sums, averages, and counts.

Question 4

Which operator is used to create a calculated column?

A. join
B. where
C. summarize
D. extend

Correct Answer: D

Explanation: The extend operator creates new calculated columns within a query.

Question 5

You need to display the number of events generated every hour. Which function should be used?

A. bin()
B. tostring()
C. parse_json()
D. countif()

Correct Answer: A

Explanation: The bin() function groups data into fixed time intervals for time-series analysis.

Question 6

Which Fabric component serves as the primary analytics engine for KQL workloads?

A. Lakehouse
B. Warehouse
C. Eventhouse
D. Dataflow Gen2

Correct Answer: C

Explanation: Eventhouses are designed for high-scale event ingestion and KQL-based analytics.

Question 7

What is the primary benefit of a materialized view?

A. Data encryption
B. Faster query performance through precomputed results
C. Reduced storage requirements
D. Automatic schema detection

Correct Answer: B

Explanation: Materialized views store precomputed query results, reducing query execution time.

Question 8

A data engineer must automatically transform incoming JSON data during ingestion. Which feature should be used?

A. Spark checkpointing
B. Eventstream routing
C. Data Activator
D. Update policies

Correct Answer: D

Explanation: Update policies automatically transform data as it is ingested into KQL tables.

Question 9

Which scenario is best suited for KQL instead of Spark Structured Streaming?

A. Large-scale machine learning pipeline
B. Deep learning model training
C. Interactive analysis of streaming telemetry data
D. Complex ETL involving hundreds of joins

Correct Answer: C

Explanation: KQL excels at real-time querying and analytics of telemetry, log, and event data.

Question 10

Which KQL operator is used to select specific columns from a dataset?

A. project
B. summarize
C. extend
D. where

Correct Answer: A

Explanation: The project operator returns only the specified columns, improving efficiency and readability.

Go to the DP-700 Exam Prep Hub main page.

DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Process data by using Spark structured streaming (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform streaming data
      --> Process data by using Spark structured streaming

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Modern analytics platforms increasingly require the ability to process data continuously as it arrives rather than waiting for scheduled batch loads. Microsoft Fabric supports this requirement through Spark Structured Streaming, a scalable and fault-tolerant stream processing engine built on Apache Spark.

For the DP-700 exam, you should understand when and how to use Spark Structured Streaming, how it differs from other real-time processing options such as Eventstreams and KQL Querysets, and how to design streaming solutions that write data into OneLake and Delta tables.

Spark Structured Streaming is commonly used when data engineers need to process streaming data with complex transformations, enrichments, joins, aggregations, and machine learning workloads while leveraging the scalability of Spark. (Microsoft Learn)

What Is Spark Structured Streaming?

Spark Structured Streaming is a stream-processing framework built on top of Apache Spark. It treats a continuous stream of incoming data as an unbounded table to which new rows are constantly appended. Developers write code using familiar DataFrame and Spark SQL operations while Spark handles the continuous execution behind the scenes. (Microsoft Learn)

Key characteristics include:

Near real-time processing
Fault tolerance
Automatic recovery
Horizontal scalability
Support for complex transformations
Integration with Delta Lake
Exactly-once processing capabilities through checkpointing and transaction logs (Microsoft Learn)

How Structured Streaming Works

The processing flow typically follows these steps:

Read data from a streaming source.
Apply transformations.
Write results to a destination.
Store checkpoints to track processing progress.
Continue processing new data as it arrives.

Common Sources

Spark Structured Streaming supports sources such as:

Azure Event Hubs
Apache Kafka
JSON files
CSV files
Parquet files
Delta tables
Eventstreams outputs (Microsoft Learn)

Common Destinations

Results can be written to:

Lakehouse Delta tables
OneLake storage
Eventhouses
Memory sinks for testing
Other supported storage locations (Microsoft Learn)

Structured Streaming in Microsoft Fabric

Within Microsoft Fabric, Structured Streaming is most commonly implemented through:

Notebooks
Spark Job Definitions
Lakehouses
Delta tables
OneLake storage

A typical architecture looks like this:

			
Azure Event Hub
       |
       v
Spark Structured Streaming
       |
       v
Delta Table in Lakehouse
       |
       v
SQL Analytics Endpoint
       |
       v
Power BI

		

This pattern enables streaming data to become queryable almost immediately after arrival. (Microsoft Learn)

Structured Streaming vs Batch Processing

Feature	Batch Processing	Structured Streaming
Data arrival	Periodic	Continuous
Processing latency	Minutes or hours	Seconds or minutes
Resource usage	Scheduled	Continuous
Typical use case	Historical reporting	Real-time analytics
Data availability	After load completes	Near real time

Use Batch Processing When:

Data changes infrequently
Overnight processing is acceptable
Real-time insights are unnecessary

Use Structured Streaming When:

IoT devices generate events continuously
Fraud detection requires immediate action
Operational dashboards need live updates
Telemetry data must be analyzed continuously

Micro-Batch Processing

A common DP-700 exam topic is understanding that Structured Streaming typically uses a micro-batch architecture.

Instead of processing every individual event separately, Spark groups events into small batches and processes them continuously.

Example:

			
Incoming Events
-----------------
10:00:00 - Event A
10:00:01 - Event B
10:00:02 - Event C
Micro-batch executes
Process A+B+C together

		

This approach provides:

Better performance
Higher throughput
Easier fault recovery
Familiar Spark execution model (Microsoft Learn)

Reading Streaming Data

Streaming ingestion begins with:

			
df = (
    spark.readStream
         .format("eventhubs")
         .load()
)

		

Important points:

readStream creates a streaming DataFrame.
Data remains continuously available.
Spark automatically detects new events.
The query remains active until stopped. (Microsoft Learn)

Writing Streaming Data

Streaming results are written using writeStream.

Example:

			
query = (
    df.writeStream
      .format("delta")
      .outputMode("append")
      .toTable("SalesEvents")
)

		

Common output modes include:

Mode	Description
Append	Only new rows written
Complete	Entire result rewritten
Update	Only changed rows written

For Fabric data engineering scenarios, Append mode is most common. (Microsoft Learn)

Delta Lake Integration

One of the most important DP-700 concepts is integrating Structured Streaming with Delta Lake.

Benefits include:

ACID transactions
Schema evolution
Time travel
Data versioning
Reliable streaming ingestion

Streaming data can be written directly into Delta tables:

			
.writeStream
.format("delta")
.toTable("Orders")

This creates a continuously updated Delta table within the Lakehouse. (Microsoft Learn)

Checkpointing

Checkpointing is critical for fault tolerance.

Example:

			
.option(
    "checkpointLocation",
    "Files/checkpoints/orders"
)

Checkpoints store:

Processed offsets
Query progress
State information

Benefits:

Prevents duplicate processing
Enables recovery after failures
Supports exactly-once processing semantics

A frequent exam scenario involves identifying missing checkpoint configurations as the root cause of duplicate or reprocessed data. (mindmeshacademy.com)

Triggers

Triggers control how often Spark processes incoming data.

Example:

			
.trigger(
    processingTime="1 minute"
)

Possible trigger strategies:

Trigger Type	Purpose
Continuous processing	Lowest latency
Processing time	Fixed intervals
Available Now	Process all available data and stop

Larger trigger intervals often improve throughput because more events are processed together. (Microsoft Learn)

Stateful vs Stateless Processing

Stateless Processing

Each event is processed independently.

Examples:

Filtering
Column selection
Simple transformations

stream.filter("temperature > 100")

Stateful Processing

Spark maintains information between batches.

Examples:

Running totals
Session windows
Stream aggregations
Deduplication

Stateful processing is more powerful but consumes additional memory and storage resources. (jumpstart.fabric.microsoft.com)

Stream Aggregations

Streaming aggregations allow continuous calculations.

Examples:

Sales totals
Device counts
Average temperatures
Transaction volumes

Example:

			
stream.groupBy("DeviceID") \
      .count()

This continuously updates counts as new events arrive.

Common Streaming Scenarios in Fabric

IoT Monitoring

Sensors continuously send readings.

Process:

			
IoT Devices
     |
Event Hub
     |
Spark Structured Streaming
     |
Lakehouse
     |
Power BI Dashboard

		

Application Telemetry

Applications send logs and metrics continuously.

Use cases:

Performance monitoring
Error tracking
Operational dashboards

Real-Time Business Analytics

Examples include:

Online sales monitoring
Inventory tracking
Customer activity analysis
Fraud detection

Structured Streaming vs Eventstreams

DP-700 often tests when to use each technology.

Requirement	Eventstreams	Structured Streaming
No-code ingestion	Yes	No
Visual design	Yes	No
Complex transformations	Limited	Excellent
Custom code	No	Yes
Machine learning integration	Limited	Excellent
Advanced Spark operations	No	Yes

Use Eventstreams for simple routing and ingestion.

Use Structured Streaming for advanced engineering workloads. (Microsoft Learn)

Production Best Practices

Use Spark Job Definitions

For production workloads, Microsoft recommends Spark Job Definitions rather than leaving notebooks running continuously. They provide better reliability and restart capabilities. (Microsoft Learn)

Configure Retry Policies

Retry policies allow automatic recovery from infrastructure failures. (Microsoft Learn)

Always Use Checkpoints

Never deploy production streaming jobs without checkpoint locations. (mindmeshacademy.com)

Optimize Partitioning

Appropriate partitioning improves throughput and downstream query performance. (Microsoft Learn)

Monitor Streaming Jobs

Use Fabric Monitoring Hub to monitor:

Input rate
Processing rate
Batch duration
Streaming query health (Microsoft Learn)

DP-700 Exam Tips

Remember these frequently tested concepts:

Structured Streaming treats streams as continuously growing tables.
readStream reads streaming data.
writeStream writes streaming data.
Delta tables are common streaming destinations.
Checkpointing enables fault tolerance.
Spark Job Definitions are preferred for production streaming workloads.
Event Hubs is a common streaming source.
Micro-batch processing is the default execution model.
Structured Streaming is preferred when complex transformations are required.
Eventstreams are often preferred for simpler ingestion scenarios.

Practice Exam Questions

Question 1

A company needs to process telemetry data from thousands of IoT devices as soon as it arrives. The solution must perform complex transformations before storing data in a Lakehouse.

Which technology should you choose?

A. Dataflow Gen2
B. Warehouse Stored Procedures
C. Spark Structured Streaming
D. Copy Activity

Correct Answer: C

Explanation: Spark Structured Streaming is designed for continuous data processing and complex transformations on streaming data.

Question 2

What is the primary purpose of a checkpoint location in Structured Streaming?

A. Increase Spark cluster size
B. Store temporary query results
C. Track processing progress and support recovery
D. Compress Delta files

Correct Answer: C

Explanation: Checkpoints store offsets and state information that allow recovery without reprocessing all data.

Question 3

Which method is used to create a streaming DataFrame?

A. readStream()
B. streamRead()
C. loadStreaming()
D. readDelta()

Correct Answer: A

Explanation: readStream() is the Spark API used to create streaming DataFrames.

Question 4

Which destination is most commonly used for Spark Structured Streaming in Microsoft Fabric?

A. Delta table in a Lakehouse
B. Excel workbook
C. Dataflow Gen2
D. Semantic model

Correct Answer: A

Explanation: Delta tables in Lakehouses are the primary streaming storage destination in Fabric.

Question 5

What execution model does Spark Structured Streaming primarily use?

A. Row-by-row execution
B. Continuous SQL polling
C. Micro-batch processing
D. Manual scheduling

Correct Answer: C

Explanation: Structured Streaming processes incoming data as small batches at regular intervals.

Question 6

Which Fabric component is recommended for running production Structured Streaming workloads?

A. Notebook only
B. Dataflow Gen2
C. Pipeline activity
D. Spark Job Definition

Correct Answer: D

Explanation: Spark Job Definitions provide improved reliability, retry policies, and production-grade execution.

Question 7

A streaming job must continuously calculate running totals by customer.

What type of processing is required?

A. Stateless processing
B. Stateful processing
C. Batch processing
D. Snapshot processing

Correct Answer: B

Explanation: Running totals require maintaining state across multiple batches.

Question 8

Which statement about Eventstreams and Structured Streaming is correct?

A. Eventstreams supports more advanced Spark transformations.
B. Structured Streaming is a no-code solution.
C. Structured Streaming supports complex custom code transformations.
D. Eventstreams requires Spark coding.

Correct Answer: C

Explanation: Structured Streaming provides full Spark capabilities and custom coding flexibility.

Question 9

What is the benefit of writing streaming data to Delta tables?

A. Eliminates storage costs
B. Prevents all schema changes
C. Converts data to CSV automatically
D. Provides ACID transactions and reliability

Correct Answer: D

Explanation: Delta Lake provides transactional consistency, schema evolution, and reliable streaming ingestion.

Question 10

A data engineer wants to process incoming events every 60 seconds instead of immediately.

Which feature should be configured?

A. Checkpointing
B. Consumer groups
C. Trigger interval
D. Data partitioning

Correct Answer: C

Explanation: Trigger intervals control how frequently Spark processes incoming streaming data.

Go to the DP-700 Exam Prep Hub main page.

DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Process data by using Eventstreams (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform streaming data
      --> Process data by using Eventstreams

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

As organizations increasingly rely on real-time analytics, the ability to ingest, process, route, and analyze streaming data has become a critical skill for data engineers. Microsoft Fabric provides Eventstreams as a low-code, scalable solution for processing streaming data within the Real-Time Intelligence workload.

For the DP-700 exam, you should understand how Eventstreams work, how they integrate with other Fabric components, how to perform basic stream processing, and when to use Eventstreams instead of alternatives such as notebooks, pipelines, or KQL databases.

What Are Eventstreams?

An Eventstream is a real-time data processing service within Microsoft Fabric that enables users to:

Ingest streaming data from various sources
Process and transform events in motion
Route data to multiple destinations
Monitor streaming pipelines visually
Build real-time analytics solutions

Eventstreams serve as the ingestion and routing layer of many Real-Time Intelligence solutions.

Conceptually:

			
Data Sources
      ↓
 Eventstream
      ↓
Processing & Routing
      ↓
Destinations

		

Eventstreams allow organizations to handle millions of events while maintaining low latency and high scalability.

Why Use Eventstreams?

Traditional batch processing waits for data to accumulate before processing.

Streaming scenarios require:

Immediate processing
Low-latency analytics
Real-time alerts
Continuous monitoring

Examples include:

IoT sensor monitoring
Website clickstream analysis
Application telemetry
Manufacturing equipment monitoring
Financial transaction processing
Security event monitoring

Eventstreams provide a managed platform for handling these requirements.

Eventstream Architecture

An Eventstream consists of three major components:

1. Sources

Sources provide incoming event data.

Common sources include:

Event Hubs
Fabric Eventhouses
Azure IoT Hub
Fabric Real-Time Hub
Custom applications
Sample streaming data

Example:

			
IoT Devices
      ↓
Azure Event Hubs
      ↓
Eventstream

		

2. Processing

After ingestion, Eventstreams can perform lightweight transformations.

Examples include:

Filtering records
Selecting columns
Enriching events
Basic data transformations
Event routing

Processing occurs while data is flowing through the stream.

3. Destinations

Processed events can be delivered to one or more destinations.

Common destinations include:

Eventhouse
KQL Database
Lakehouse
Activator
Custom endpoints

Example:

			
Eventstream
    ↓
 ┌─────────┬─────────┬─────────┐
 │Lakehouse│Eventhouse│Activator│
 └─────────┴─────────┴─────────┘

		

One incoming stream can be delivered to multiple destinations simultaneously.

Eventstreams and Real-Time Intelligence

Eventstreams are a foundational component of Fabric Real-Time Intelligence.

A typical architecture may include:

			
IoT Devices
      ↓
Eventstream
      ↓
Eventhouse
      ↓
KQL Queries
      ↓
Dashboards

		

In this architecture:

Eventstream ingests data.
Eventhouse stores data.
KQL analyzes data.
Dashboards visualize results.

Common Eventstream Sources

Azure Event Hubs

One of the most common production sources.

Use when:

High-volume streaming data exists
Enterprise-scale ingestion is required
External systems already publish events

Azure IoT Hub

Designed specifically for IoT devices.

Examples:

Manufacturing sensors
Smart buildings
Connected vehicles

Real-Time Hub

Fabric Real-Time Hub provides a centralized location for discovering and connecting streaming data sources.

Benefits include:

Simplified discovery
Easy integration
Centralized event management

Eventstream Processing Capabilities

Eventstreams support several lightweight transformation capabilities.

Filtering

Filter unwanted records before storage.

Example:

Only process temperatures above 80°F.

Input:

			
Device A: 75
Device B: 84
Device C: 81

Output:

			
Device B: 84
Device C: 81

Filtering reduces storage and processing costs.

Column Selection

Keep only required fields.

Input:

			
DeviceID
Temperature
Location
BatteryLevel
Timestamp

		

Output:

			
DeviceID
Temperature
Timestamp

This reduces data volume.

Data Enrichment

Additional information can be added to streaming events.

Example:

			
Incoming Event:
DeviceID = 100
Enriched Event:
DeviceID = 100
Region = East
Facility = Orlando

		

Enrichment improves downstream analytics.

Routing Events

One of the most important Eventstream features is routing.

A single incoming stream can be sent to multiple destinations.

Example:

			
Telemetry Stream
        ↓
   Eventstream
        ↓
 ┌────────┬─────────┬─────────┐
 │Lakehouse│Eventhouse│Activator│
 └────────┴─────────┴─────────┘

		

This enables:

Historical storage
Real-time analytics
Automated actions

from the same stream.

Eventstream Destinations

Eventhouse

Best for:

KQL analytics
Real-time dashboards
Time-series analysis

Often the primary destination in Real-Time Intelligence solutions.

Lakehouse

Best for:

Historical retention
Data science
Long-term storage
Delta table analytics

Commonly used alongside Eventhouse.

Activator

Used to trigger actions based on conditions.

Examples:

Send alerts
Trigger workflows
Notify users

Example:

			
Temperature > 100°F
       ↓
Send Alert

Eventstream Monitoring

Fabric provides monitoring capabilities for Eventstreams.

Metrics include:

Throughput
Incoming events
Failed events
Processing latency
Destination status

Monitoring helps identify:

Bottlenecks
Connection issues
Data quality problems

Eventstreams vs Pipelines

This comparison is important for the DP-700 exam.

Feature	Eventstream	Pipeline
Real-time processing	Yes	No
Streaming data	Yes	No
Batch processing	Limited	Yes
Continuous execution	Yes	No
Scheduling	No	Yes
Data movement	Yes	Yes

Use Eventstreams When

Data arrives continuously
Low latency is required
Real-time monitoring is needed

Use Pipelines When

Batch processing is required
Scheduled execution is needed
ETL orchestration is required

Eventstreams vs Notebooks

Feature	Eventstream	Notebook
Low-code	Yes	No
Streaming ingestion	Yes	Possible
Complex transformations	Limited	Extensive
Spark processing	No	Yes
Machine learning	No	Yes

Use Eventstreams

For simple streaming ingestion and routing.

Use Notebooks

For advanced Spark transformations and machine learning workloads.

Eventstreams vs Eventhouse

Candidates often confuse these services.

Eventstream

Focuses on:

Ingestion
Processing
Routing

Eventhouse

Focuses on:

Storage
Querying
Analytics

A common architecture uses both together.

			
Eventstream
      ↓
Eventhouse
      ↓
KQL Queries

		

Best Practices

Filter Early

Remove unnecessary events before storage.

Benefits:

Lower storage costs
Faster queries
Reduced processing requirements

Route Once, Consume Many

Instead of duplicating ingestion pipelines, use one Eventstream and multiple destinations.

Benefits:

Simpler architecture
Lower maintenance effort

Monitor Throughput

Regularly review:

Event ingestion rates
Failed events
Processing latency

Separate Real-Time and Historical Analytics

A common architecture is:

			
Eventstream
      ↓
 ┌──────────┬──────────┐
 │Eventhouse│Lakehouse │
 └──────────┴──────────┘

		

Eventhouse supports operational analytics while Lakehouse supports historical analysis.

DP-700 Exam Tips

Remember the following:

Eventstreams are designed for real-time data ingestion and routing.
Eventstreams consist of sources, processing, and destinations.
Eventstreams commonly feed Eventhouses.
Multiple destinations can receive the same stream.
Eventstreams support filtering, selection, and enrichment.
Eventstreams are not replacements for notebooks.
Pipelines are primarily for batch orchestration.
Eventhouse stores and analyzes streaming data.
Activator can trigger actions from streaming events.
Eventstreams are a key component of Fabric Real-Time Intelligence architectures.

Practice Exam Questions

Question 1

A company receives telemetry from thousands of IoT devices every second. The data must be processed immediately and sent to an Eventhouse.

Which Fabric component should be used?

A. Eventstream
B. Dataflow Gen2
C. Warehouse
D. Deployment Pipeline

Correct Answer: A

Explanation:
Eventstreams are designed specifically for real-time ingestion, processing, and routing of streaming data.

Question 2

Which component of an Eventstream receives incoming events?

A. Destination
B. Source
C. Activator
D. Eventhouse

Correct Answer: B

Explanation:
Sources are responsible for providing incoming streaming data to the Eventstream.

Question 3

A data engineer wants to remove all records where temperature is below 70°F before storing the data.

Which Eventstream capability should be used?

A. Mirroring
B. Aggregation
C. Filtering
D. Scheduling

Correct Answer: C

Explanation:
Filtering removes unwanted records before they reach downstream destinations.

Question 4

Which destination is best suited for real-time KQL analytics?

A. Warehouse
B. Notebook
C. Dataflow Gen2
D. Eventhouse

Correct Answer: D

Explanation:
Eventhouse is optimized for real-time analytics and KQL querying.

Question 5

A company wants the same streaming data to be stored historically and analyzed in real time.

What should be done?

A. Create two separate Eventstreams
B. Route the Eventstream to both a Lakehouse and an Eventhouse
C. Export the data twice
D. Use Dataflow Gen2

Correct Answer: B

Explanation:
Eventstreams can send data to multiple destinations simultaneously.

Question 6

Which Fabric service can trigger alerts based on conditions detected in streaming data?

A. Pipeline
B. Activator
C. Warehouse
D. Notebook

Correct Answer: B

Explanation:
Activator can generate notifications and actions based on event conditions.

Question 7

Which statement best describes Eventstreams?

A. Primarily used for batch ETL scheduling
B. Primarily used for dashboard creation
C. Primarily used for real-time ingestion and routing
D. Primarily used for SQL warehousing

Correct Answer: C

Explanation:
Eventstreams specialize in streaming ingestion, lightweight processing, and routing.

Question 8

Which service is generally preferred for complex Spark-based transformations?

A. Eventstream
B. Activator
C. Eventhouse
D. Notebook

Correct Answer: D

Explanation:
Notebooks provide extensive Spark and PySpark transformation capabilities that exceed Eventstream processing functionality.

Question 9

What is a major benefit of routing a stream to multiple destinations?

A. Eliminates all storage costs
B. Allows different workloads to consume the same stream simultaneously
C. Removes the need for Eventhouse
D. Prevents data retention

Correct Answer: B

Explanation:
Multiple destinations allow operational analytics, historical storage, and alerting from the same data stream.

Question 10

Which statement accurately compares Eventstreams and pipelines?

A. Pipelines are optimized for continuous streaming ingestion.
B. Eventstreams are primarily used for batch scheduling.
C. Both services are identical.
D. Eventstreams are optimized for real-time processing, while pipelines are optimized for batch orchestration.

Correct Answer: D

Explanation:
Eventstreams handle continuously arriving data, while pipelines are designed for orchestrated batch processing and scheduled workflows.

Go to the DP-700 Exam Prep Hub main page.

DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Choose between Query Acceleration for OneLake shortcuts and standard OneLake shortcuts in Real-Time Intelligence (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform streaming data
      --> Choose between Query Acceleration for OneLake shortcuts and standard OneLake shortcuts in Real-Time Intelligence

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Microsoft Fabric provides multiple ways to access data stored in OneLake from Real-Time Intelligence workloads such as Eventhouses and KQL databases. One of the most important design decisions for data engineers is determining whether to use:

Standard OneLake shortcuts
Query-accelerated OneLake shortcuts

Understanding the differences between these options is essential for the DP-700 exam because they directly affect performance, cost, latency, storage consumption, and analytics architecture.

This article explains how each option works, when to use them, their limitations, and the decision-making criteria you should understand for the exam.

Understanding OneLake Shortcuts

A OneLake shortcut is a virtual reference to data stored elsewhere. Instead of copying data, the shortcut points to an existing data source. This allows multiple Fabric experiences to access the same data without creating duplicate copies. (Microsoft Learn)

For example:

A Lakehouse contains sales data.
An Eventhouse creates a shortcut to that data.
Queries can access the data through the shortcut.
The original data remains in its source location.

Benefits include:

No data duplication
Reduced storage costs
Single source of truth
Simplified data management
Faster implementation

Standard OneLake Shortcuts

A standard OneLake shortcut allows Real-Time Intelligence workloads to query external data directly from OneLake without ingesting it into the Eventhouse. (Microsoft Learn)

How It Works

When a query executes:

Eventhouse accesses the shortcut.
Data is retrieved from the source Delta table.
Results are returned to the query.

No additional indexing or caching is performed.

Advantages

Minimal setup effort
No duplicated storage
Lower cost
Immediate access to existing data
Suitable for infrequent queries

Disadvantages

Slower query performance
Higher query latency
External storage access required during execution
Limited optimization opportunities

Query Acceleration for OneLake Shortcuts

Query Acceleration is a feature in Real-Time Intelligence that improves query performance against OneLake shortcut data by automatically caching and indexing selected data. (Video2 Skills Academy)

Instead of repeatedly reading Delta files from storage, Fabric creates optimized structures that significantly improve performance.

How It Works

When acceleration is enabled:

A shortcut is created.
Fabric indexes the data.
Fabric caches data based on the configured retention period.
Queries use optimized structures instead of repeatedly scanning raw files. (Microsoft Learn)

The experience becomes similar to querying native Eventhouse data.

Query Acceleration Architecture

Without acceleration:

			
Delta Table
      ↓
OneLake Shortcut
      ↓
Query Reads Files Directly

		

With acceleration:

			
Delta Table
      ↓
OneLake Shortcut
      ↓
Indexing and Caching
      ↓
High-Performance Queries

		

Performance Comparison

Characteristic	Standard Shortcut	Query Accelerated Shortcut
Data duplication	No	No
Caching	No	Yes
Indexing	No	Yes
Query latency	Higher	Lower
Large-scale analytics	Moderate	Excellent
Cost	Lower	Higher
Setup complexity	Low	Moderate

When to Use Standard OneLake Shortcuts

Choose standard shortcuts when:

Query Frequency is Low

If users only occasionally access the data, acceleration may not provide sufficient value.

Example:

Monthly compliance reports
Ad hoc investigations
Occasional auditing

Cost Optimization is Critical

Since acceleration introduces caching and indexing costs, standard shortcuts are often preferred for budget-sensitive workloads.

Data Volumes are Small

Smaller datasets generally perform well enough without acceleration.

When to Use Query Acceleration

Choose query acceleration when:

High Query Volume Exists

Examples:

Interactive dashboards
Continuous monitoring
Frequent analytics workloads

Large Delta Tables Are Queried

Large historical datasets often benefit significantly from acceleration.

Real-Time and Historical Data Must Be Combined

A common Real-Time Intelligence pattern involves:

Streaming data arriving in Eventhouse
Historical data stored in OneLake

Query acceleration enables efficient joins between both datasets. (Video2 Skills Academy)

Example:

			
Live Sensor Stream
        +
Historical Equipment Data
        =
Real-Time Analytics

		

Dimension Data Must Be Joined Frequently

Organizations often mirror dimension data into OneLake and then use accelerated shortcuts for enrichment and lookup operations. (Video2 Skills Academy)

Configuring Query Acceleration

Acceleration can be enabled:

During shortcut creation
After shortcut creation through Data Policies settings (Microsoft Learn)

Administrators can also define:

Number of cached days
Retention period
Acceleration policies

The caching period determines how much data remains optimized for high-performance access. (Microsoft Learn)

Caching Period Considerations

The caching period directly impacts:

Query performance
Storage consumption
Cost

Example:

Cached Period	Typical Use Case
7 days	Operational monitoring
30 days	Business analytics
90 days	Historical trend analysis

Longer periods improve performance across larger time ranges but increase storage costs.

Cost Considerations

This topic frequently appears in architecture-based exam questions.

Standard Shortcuts

Costs include:

Storage
Query processing

No additional acceleration charges apply.

Query Acceleration

Additional costs include:

Cached data storage
Index maintenance
Acceleration processing (Video2 Skills Academy)

The tradeoff is:

			
Higher Cost
      ↓
Much Better Performance

Limitations of Query Acceleration

Candidates should understand major limitations.

Examples include: (Video2 Skills Academy)

Materialized views are not supported.
Update policies are not supported.
External tables with extremely large file counts may experience reduced effectiveness.
Certain Delta table schema changes may require reacceleration.
Some advanced Delta features may require disabling and re-enabling acceleration.

Decision Framework for the Exam

A useful exam strategy:

Choose Standard Shortcuts When

Cost is the highest priority.
Data is queried infrequently.
Data volume is moderate.
Performance requirements are relaxed.

Choose Query Acceleration When

Performance is critical.
Queries occur frequently.
Large datasets are analyzed.
Historical and streaming data are combined.
Interactive analytics workloads exist.

DP-700 Exam Tips

Remember These Key Points

OneLake shortcuts avoid data duplication.
Standard shortcuts access data directly.
Query acceleration adds indexing and caching.
Query acceleration improves performance but increases cost.
Accelerated shortcuts are ideal for frequent analytical queries.
Standard shortcuts are ideal for occasional access scenarios.
Query acceleration is especially valuable when combining streaming and historical datasets.
Cached retention periods directly affect cost and performance.
Accelerated shortcuts behave like external tables and inherit some external table limitations.
The exam often focuses on choosing the most cost-effective versus highest-performance solution.

Practice Exam Questions

Question 1

A company uses Eventhouse to analyze telemetry data. Historical data resides in OneLake and is queried thousands of times per day. Query performance is poor.

What should you implement?

A. Dataflows Gen2
B. Query acceleration on the OneLake shortcut
C. Warehouse mirroring
D. Notebook scheduling

Correct Answer: B

Explanation:
Query acceleration adds indexing and caching that significantly improves query performance for frequently accessed shortcut data. (Video2 Skills Academy)

Question 2

What is the primary benefit of a standard OneLake shortcut?

A. Eliminates all query latency
B. Automatically indexes data
C. Provides access to data without duplication
D. Creates materialized views

Correct Answer: C

Explanation:
Shortcuts reference existing data rather than copying it, allowing a single source of truth. (Microsoft Learn)

Question 3

A solution prioritizes the lowest possible storage and acceleration costs. Data is queried only once per month.

Which option should be selected?

A. Query-accelerated shortcut
B. Materialized view
C. Standard OneLake shortcut
D. Native Eventhouse ingestion

Correct Answer: C

Explanation:
When query frequency is very low, the additional acceleration costs are generally not justified.

Question 4

What additional capability does query acceleration provide?

A. Encryption
B. Data mirroring
C. Row-level security
D. Caching and indexing

Correct Answer: D

Explanation:
Query acceleration improves performance through indexing and caching. (Video2 Skills Academy)

Question 5

Which scenario most strongly justifies query acceleration?

A. Small dataset queried quarterly
B. Development environment testing
C. Large historical dataset used in interactive dashboards
D. One-time data migration

Correct Answer: C

Explanation:
Interactive dashboards require low latency and frequent queries, making acceleration highly beneficial.

Question 6

What happens to the source data when a OneLake shortcut is created?

A. It is copied into Eventhouse
B. It is archived
C. It is compressed
D. It remains in its original location

Correct Answer: D

Explanation:
A shortcut is only a reference to the original data source. (Microsoft Learn)

Question 7

An engineer wants to join streaming Eventhouse data with historical OneLake data while maintaining high query performance.

Which approach should be recommended?

A. Query-accelerated shortcut
B. Dataflow Gen2
C. Warehouse endpoint
D. Manual exports

Correct Answer: A

Explanation:
One of the primary use cases for query acceleration is combining streaming and historical data efficiently. (Video2 Skills Academy)

Question 8

What configuration primarily controls how much accelerated data remains cached?

A. Workspace role assignments
B. Retention and caching period settings
C. Lakehouse schema definitions
D. Fabric tenant settings

Correct Answer: B

Explanation:
Administrators specify how many days of data are retained in the acceleration cache. (Microsoft Learn)

Question 9

Which statement about accelerated shortcuts is true?

A. They always cost less than standard shortcuts.
B. They require data duplication.
C. They can improve performance through cached and indexed data.
D. They eliminate storage requirements.

Correct Answer: C

Explanation:
Acceleration works by indexing and caching data while still avoiding data duplication. (Video2 Skills Academy)

Question 10

A company needs the fastest possible query performance against frequently accessed OneLake data and is willing to accept additional cost.

Which option should be chosen?

A. Standard OneLake shortcut
B. Manual exports to CSV
C. Dataflow Gen2
D. Query-accelerated OneLake shortcut

Correct Answer: D

Explanation:
Query acceleration is specifically designed to maximize query performance by using caching and indexing mechanisms. (Video2 Skills Academy)

Go to the DP-700 Exam Prep Hub main page.

DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Choose between native tables and OneLake shortcuts in Real-Time Intelligence (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform streaming data
      --> Choose between native tables and OneLake shortcuts in Real-Time Intelligence

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

One of the key design decisions when building real-time analytics solutions in Microsoft Fabric is determining where data should reside and how it should be accessed. Within Real-Time Intelligence, data engineers frequently encounter scenarios where they must choose between:

Native Tables in Eventhouse/KQL databases
OneLake Shortcuts to data stored elsewhere

Understanding the differences between these approaches is important for the DP-700 exam because the choice impacts:

Query performance
Data latency
Storage costs
Data governance
Data duplication
Maintenance complexity

A successful data engineer must understand when to ingest data directly into Real-Time Intelligence and when to reference existing data through shortcuts.

Understanding Real-Time Intelligence

Real-Time Intelligence is Microsoft Fabric’s solution for ingesting, analyzing, and acting upon streaming and operational data.

Key components include:

Eventstream
Eventhouse
KQL Databases
Data Activator
Real-Time Dashboards

Data stored within Eventhouse and KQL databases can come from multiple sources:

Direct streaming ingestion
Batch ingestion
External storage systems
OneLake data sources

This is where the choice between native tables and OneLake shortcuts becomes important.

What Are Native Tables?

Native tables are physical tables stored directly inside a KQL database or Eventhouse.

When data is ingested into Real-Time Intelligence, it is written into these tables and becomes part of the Eventhouse storage engine.

Characteristics of Native Tables

Native tables:

Physically store data
Support extremely fast query performance
Are optimized for time-series analytics
Support continuous streaming ingestion
Provide low-latency access
Support update policies and materialized views
Enable advanced KQL analytics

Native Table Architecture

			
Event Source
      ↓
Eventstream
      ↓
Native Table
      ↓
KQL Queries
      ↓
Dashboards / Analytics

		

Data resides directly within the Eventhouse environment.

Advantages of Native Tables

Highest Query Performance

Because data is physically stored in the Eventhouse engine, query execution is highly optimized.

Benefits include:

Faster aggregations
Faster filtering
Lower latency
Better concurrency

Optimized for Streaming Workloads

Native tables are specifically designed for:

High ingestion rates
Continuous event streams
Telemetry data
Operational analytics

Support for Advanced Features

Native tables support:

Materialized views
Update policies
Data retention policies
Cached query execution
Time-series functions

Lower Query Latency

Real-time dashboards often require results within seconds.

Native tables generally provide the lowest latency.

Disadvantages of Native Tables

Data Duplication

The same data may already exist elsewhere:

Lakehouse
Warehouse
ADLS Gen2
Other databases

Ingesting into native tables creates another copy.

Increased Storage Costs

More copies of data mean:

More storage consumption
Additional retention management

Additional Ingestion Processing

Data must be:

Moved
Loaded
Managed

before it becomes available.

What Are OneLake Shortcuts?

A OneLake shortcut provides a virtual reference to data stored elsewhere.

Rather than copying data into Eventhouse, Real-Time Intelligence accesses the existing data through the shortcut.

Shortcut Concept

Instead of:

Source → Copy → Eventhouse

You get:

Source → OneLake Shortcut → Query

No physical duplication occurs.

Supported Sources

Shortcuts can reference:

Fabric Lakehouses
Fabric Warehouses
Azure Data Lake Storage Gen2
Amazon S3
Other supported storage locations

Characteristics of OneLake Shortcuts

Shortcuts:

Avoid copying data
Provide a single source of truth
Reduce storage costs
Simplify governance
Enable data reuse

Advantages of OneLake Shortcuts

Eliminate Data Duplication

One of the biggest advantages.

Instead of storing multiple copies:

			
One Source
    ↓
Multiple Consumers

All consumers access the same data.

Lower Storage Costs

Since data is not duplicated:

Less storage consumption
Lower management overhead

Faster Data Availability

No ingestion process is required.

Data becomes accessible immediately after the shortcut is created.

Improved Governance

Governance becomes easier because:

Data remains in one location
Policies remain centralized
Data lineage remains clearer

Supports the One Copy Vision

OneLake is built around the principle of:

“One copy of data for the entire organization.”

Shortcuts are a key enabler of this strategy.

Disadvantages of OneLake Shortcuts

Potentially Higher Query Latency

Because data is not stored locally:

Queries may require additional access steps
Performance can be slower than native tables

Limited Optimization

Some advanced Eventhouse optimization capabilities are most effective with native data.

Examples include:

Materialized views
Update policies
Streaming ingestion optimizations

Dependency on Source Availability

If the source becomes unavailable:

Queries may fail
Performance may degrade

Native tables do not have this dependency.

When to Choose Native Tables

Choose native tables when:

Real-Time Performance Is Critical

Examples:

Monitoring dashboards
Security analytics
Fraud detection
Manufacturing telemetry

Continuous Streaming Ingestion Exists

Examples:

IoT sensors
Application logs
Device telemetry

High Query Volumes Are Expected

Examples:

Enterprise dashboards
Operational reporting

Advanced KQL Features Are Required

Examples:

Materialized views
Update policies
Retention policies

When to Choose OneLake Shortcuts

Choose shortcuts when:

Data Already Exists in OneLake

Avoid creating unnecessary copies.

Storage Costs Must Be Minimized

Shortcuts reduce storage requirements.

Data Sharing Is Important

Multiple teams can access the same dataset.

Data Is Primarily Historical

Examples:

Historical archives
Reference datasets
Slowly changing datasets

Governance Is a Priority

Maintaining a single source of truth simplifies compliance and governance efforts.

Comparing Native Tables and OneLake Shortcuts

Feature	Native Tables	OneLake Shortcuts
Physical storage	Yes	No
Data duplication	Yes	No
Storage cost	Higher	Lower
Query performance	Highest	Good
Streaming ingestion	Excellent	Not primary purpose
Advanced KQL features	Full support	Limited scenarios
Data governance	More complex	Simpler
Single source of truth	No	Yes
Real-time analytics	Best choice	Suitable in some cases
Historical data access	Good	Excellent

Common DP-700 Exam Scenarios

Scenario 1

A manufacturing company ingests millions of telemetry events every minute and requires dashboards that refresh within seconds.

Best Choice: Native Tables

Reason:

Maximum ingestion performance
Lowest query latency

Scenario 2

An organization already stores enterprise sales data in a Fabric Lakehouse and wants Eventhouse users to analyze it without creating another copy.

Best Choice: OneLake Shortcut

Reason:

Eliminates duplication
Supports centralized governance

Scenario 3

A security operations center performs continuous threat monitoring using KQL.

Best Choice: Native Tables

Reason:

Optimized for streaming analytics
Fast query response times

Scenario 4

A data engineering team needs occasional access to historical archive data stored in ADLS Gen2.

Best Choice: OneLake Shortcut

Reason:

No need to ingest large historical datasets
Lower storage costs

Decision Framework

Ask the following questions:

Is the data arriving continuously?

If yes → Native Tables.

Is ultra-low latency required?

If yes → Native Tables.

Does the data already exist in OneLake?

If yes → Consider OneLake Shortcuts.

Is avoiding duplication important?

If yes → OneLake Shortcuts.

Are advanced KQL optimization features required?

If yes → Native Tables.

DP-700 Exam Tips

Remember these key distinctions:

Native tables physically store data inside Eventhouse.
Native tables provide the highest performance.
Native tables are ideal for streaming ingestion.
OneLake shortcuts reference data without copying it.
Shortcuts support the One Copy vision of OneLake.
Shortcuts reduce storage costs.
Native tables are preferred when low-latency analytics is critical.
Shortcuts are preferred when data already exists elsewhere and duplication should be avoided.
Exam questions often focus on balancing performance versus storage and governance.

Practice Exam Questions

Question 1

A company requires sub-second analytics on continuously arriving IoT telemetry data in Eventhouse.

Which storage approach should be selected?

A. OneLake shortcut to a Lakehouse
B. OneLake shortcut to ADLS Gen2
C. Native table
D. Dataflow Gen2

Answer: C

Explanation:
Native tables provide the lowest latency and are optimized for continuous streaming ingestion and real-time analytics.

Question 2

An organization already stores customer history in a Fabric Lakehouse and wants Eventhouse users to analyze the data without creating additional copies.

Which option should be used?

A. Native table
B. OneLake shortcut
C. Eventstream ingestion
D. Data Activator

Answer: B

Explanation:
OneLake shortcuts allow access to existing data without physically copying it into Eventhouse.

Question 3

What is the primary advantage of using OneLake shortcuts?

A. Faster ingestion speeds
B. Automatic materialized views
C. Lower query latency
D. Elimination of data duplication

Answer: D

Explanation:
Shortcuts provide virtual access to data and eliminate the need to create additional copies.

Question 4

Which feature is most strongly associated with native tables?

A. Single source of truth
B. External data access
C. Physical storage within Eventhouse
D. Reduced storage costs

Answer: C

Explanation:
Native tables physically store data within Eventhouse and are optimized for real-time analytics.

Question 5

A team wants to minimize storage costs while analyzing historical datasets already stored in OneLake.

Which option is best?

A. Native tables
B. OneLake shortcuts
C. Spark cache tables
D. Temporary KQL tables

Answer: B

Explanation:
Shortcuts allow direct access to existing data without storing another copy.

Question 6

Which scenario most strongly favors native tables?

A. Historical archive access
B. Shared enterprise data reuse
C. High-volume streaming telemetry analytics
D. Storage cost reduction

Answer: C

Explanation:
Native tables are designed for continuous ingestion and high-performance real-time analytics.

Question 7

A data engineer wants to support the OneLake principle of maintaining a single copy of organizational data.

Which option best aligns with this goal?

A. Native tables
B. Materialized views
C. Streaming ingestion
D. OneLake shortcuts

Answer: D

Explanation:
Shortcuts are specifically designed to support OneLake’s single-copy architecture.

Question 8

Which statement about native tables is true?

A. They never store data physically.
B. They generally provide better query performance than shortcuts.
C. They require external storage systems.
D. They cannot be queried with KQL.

Answer: B

Explanation:
Because the data is stored directly inside Eventhouse, native tables typically deliver the highest performance.

Question 9

A company wants to use advanced KQL features such as update policies and materialized views on streaming data.

Which approach should be selected?

A. OneLake shortcut
B. Warehouse shortcut
C. Native table
D. Dataflow Gen2

Answer: C

Explanation:
Advanced Eventhouse optimization features are most commonly associated with native tables.

Question 10

Which factor most commonly drives the decision to use a OneLake shortcut instead of a native table?

A. Requirement for lowest latency analytics
B. Requirement for continuous event ingestion
C. Requirement for materialized views
D. Requirement to avoid storing duplicate copies of data

Answer: D

Explanation:
The primary benefit of OneLake shortcuts is enabling data access without physically duplicating data, reducing storage costs and simplifying governance.

Go to the DP-700 Exam Prep Hub main page.

DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Choose an appropriate streaming engine (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform streaming data
      --> Choose an appropriate streaming engine

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Overview

Modern analytics solutions increasingly rely on the ability to process data as it is generated rather than waiting for scheduled batch loads. Streaming data enables organizations to react to events in near real time, support operational analytics, monitor systems, detect anomalies, and power intelligent applications.

In Microsoft Fabric, selecting the appropriate streaming engine is a critical design decision. The DP-700 exam expects candidates to understand the strengths, limitations, and ideal use cases of the various streaming technologies available in Fabric and to choose the most appropriate option based on business requirements.

This article explores the major streaming engines and technologies within Microsoft Fabric, how they compare, and when to use each one.

What Is Streaming Data?

Streaming data is data that arrives continuously from sources such as:

IoT devices
Sensors
Application logs
Clickstream events
Social media feeds
Financial transactions
Manufacturing equipment
Website activity
Real-time telemetry

Unlike batch processing, where data is collected and processed periodically, streaming systems process data as events arrive.

Common requirements include:

Low-latency processing
Real-time dashboards
Event detection
Alert generation
Continuous data ingestion
Streaming analytics

Streaming Technologies in Microsoft Fabric

The primary streaming technologies that data engineers encounter in Fabric include:

Technology	Primary Purpose
Eventstream	Real-time event ingestion and routing
Eventhouse	Real-time analytics using KQL
KQL Database	High-performance streaming analytics
Real-Time Intelligence	End-to-end real-time analytics platform
Spark Structured Streaming	Large-scale streaming transformations
Data Activator	Event-driven actions and alerts
Pipelines	Scheduled orchestration (not true streaming)

Understanding when to use each is essential for the exam.

Eventstream

What Is Eventstream?

Eventstream is Fabric’s low-code real-time ingestion service.

It captures, transforms, filters, and routes streaming events from multiple sources to multiple destinations.

Think of Eventstream as the ingestion layer of a streaming architecture.

Common Sources

Eventstream can ingest data from:

Azure Event Hubs
Kafka endpoints
Fabric events
IoT sources
Real-time telemetry systems
Custom event producers

Common Destinations

Eventstream can send data to:

Eventhouse
KQL Databases
Lakehouses
Custom destinations
Activator

Best Use Cases

Choose Eventstream when:

Events must be continuously ingested
Minimal coding is desired
Data routing is required
Multiple downstream consumers need the same events
Building real-time analytics solutions

Exam Tip

If a scenario focuses on ingesting and routing real-time events, Eventstream is usually the best answer.

Eventhouse

What Is Eventhouse?

Eventhouse is a Real-Time Intelligence component optimized for storing and analyzing streaming data.

It is built on Kusto technology and provides:

High ingestion rates
Near real-time analytics
Time-series analysis
Log analytics
Event exploration

Key Characteristics

Optimized for append-only data
Supports KQL
Fast query performance
Near real-time visibility
Massive scalability

Best Use Cases

Use Eventhouse when:

Large volumes of events arrive continuously
Log analytics is required
Telemetry analysis is needed
Operational dashboards require low latency

Examples:

Website activity monitoring
Application diagnostics
Manufacturing telemetry
Security monitoring

KQL Databases

What Is a KQL Database?

A KQL database is the storage and query engine behind many real-time solutions.

It uses Kusto Query Language (KQL) and is highly optimized for:

Streaming ingestion
Log analytics
Time-series data
Event correlation

Advantages

Extremely fast analytical queries
Handles high ingestion volumes
Rich time-series functions
Powerful aggregation capabilities

Best Use Cases

Choose KQL databases when:

Event analysis is the primary objective
Massive event volumes exist
Time-based analysis is required
Operational monitoring is needed

Spark Structured Streaming

What Is Structured Streaming?

Spark Structured Streaming enables continuous processing using Apache Spark.

Unlike Eventstream and Eventhouse, Spark streaming is developer-focused and code-driven.

Supported languages include:

PySpark
Scala
Spark SQL

Capabilities

Spark Structured Streaming supports:

Complex transformations
Data enrichment
Machine learning integration
Streaming joins
Stateful processing
Advanced business logic

Best Use Cases

Choose Spark Structured Streaming when:

Complex transformations are required
Large-scale processing is needed
Machine learning must be integrated
Events must be joined with reference datasets
Custom code is acceptable

Examples:

Fraud detection
Customer behavior analytics
Streaming feature engineering
Predictive maintenance

Exam Tip

If a scenario requires advanced coding and transformation logic, Spark Structured Streaming is often the correct answer.

Real-Time Intelligence

What Is Real-Time Intelligence?

Real-Time Intelligence is Fabric’s complete platform for handling real-time data workloads.

It combines:

Eventstream
Eventhouse
KQL Databases
Data Activator
Real-time dashboards

Benefits

Provides:

End-to-end streaming architecture
Real-time monitoring
Event processing
Alerting
Operational analytics

Best Use Cases

Use Real-Time Intelligence when an organization needs:

Comprehensive streaming analytics
Operational dashboards
Real-time monitoring
Event-driven insights

Data Activator

What Is Data Activator?

Data Activator monitors events and automatically takes actions when specified conditions occur.

Examples include:

Sending emails
Triggering workflows
Generating notifications
Creating alerts

Example

If machine temperature exceeds 90°C:

Generate an alert
Notify engineers
Open a support ticket

Best Use Cases

Choose Data Activator when:

Business users need alerts
Event-driven automation is required
Low-code monitoring is desired

Pipelines Are Not Streaming Engines

A common DP-700 exam trap is confusing pipelines with streaming solutions.

Pipelines:

Execute scheduled workloads
Orchestrate activities
Handle batch data movement

Pipelines do NOT provide continuous event processing.

Appropriate Pipeline Scenarios

Daily data loads
Weekly ETL jobs
Scheduled orchestration
Batch transformations

Inappropriate Pipeline Scenarios

Second-by-second monitoring
Real-time alerts
Continuous event processing

Selecting the Appropriate Streaming Engine

Scenario 1: IoT Sensor Telemetry

Requirements:

Millions of sensor events
Real-time monitoring
Fast analytics

Best choice:

Eventstream + Eventhouse

Scenario 2: Fraud Detection

Requirements:

Stream transactions
Apply advanced business rules
Perform enrichment joins

Best choice:

Spark Structured Streaming

Scenario 3: Website Log Analysis

Requirements:

Continuous ingestion
Fast aggregations
Time-series analysis

Best choice:

KQL Database/Eventhouse

Scenario 4: Equipment Failure Alerts

Requirements:

Detect threshold breaches
Notify operators

Best choice:

Data Activator

Scenario 5: Enterprise Real-Time Analytics Platform

Requirements:

Complete streaming solution
Dashboards
Alerts
Analytics

Best choice:

Real-Time Intelligence

Comparison of Streaming Engines

Requirement	Recommended Technology
Event ingestion	Eventstream
Event routing	Eventstream
Real-time analytics	Eventhouse
Log analytics	KQL Database
Time-series analysis	KQL Database
Complex transformations	Spark Structured Streaming
Machine learning on streams	Spark Structured Streaming
Alerts and notifications	Data Activator
Complete real-time platform	Real-Time Intelligence
Scheduled ETL	Pipelines

DP-700 Exam Tips

Remember these key distinctions:

Eventstream = ingestion and routing.
Eventhouse = real-time storage and analytics.
KQL Database = high-performance event analytics.
Spark Structured Streaming = advanced code-based processing.
Data Activator = alerts and automated actions.
Pipelines = orchestration, not streaming.
Real-Time Intelligence = end-to-end streaming solution.

Many exam questions focus on matching business requirements to the correct streaming technology.

Practice Exam Questions

Question 1

A company needs to ingest streaming telemetry from thousands of IoT devices and route the data to multiple downstream consumers.

Which Fabric component should be used?

A. Data Activator
B. Eventstream
C. Pipeline
D. Notebook

Answer: B

Explanation:
Eventstream is specifically designed for real-time event ingestion and routing. Data Activator generates actions, pipelines handle batch orchestration, and notebooks perform transformations rather than ingestion.

Question 2

A solution requires advanced stream processing with custom Python code, joins against reference datasets, and machine learning inference.

Which technology should be selected?

A. Eventhouse
B. Spark Structured Streaming
C. KQL Database
D. Data Activator

Answer: B

Explanation:
Spark Structured Streaming supports complex transformations, enrichment, stateful processing, and machine learning integration through PySpark.

Question 3

A team needs extremely fast analytics over continuously arriving log data and plans to use KQL.

Which storage engine is most appropriate?

A. KQL Database
B. Dataflow Gen2
C. Warehouse
D. Pipeline

Answer: A

Explanation:
KQL databases are optimized for streaming ingestion, time-series analysis, and log analytics.

Question 4

A business user wants automatic notifications whenever inventory levels fall below a threshold.

Which Fabric component is best suited?

A. Eventstream
B. Notebook
C. Data Activator
D. Pipeline

Answer: C

Explanation:
Data Activator monitors data conditions and triggers automated actions such as alerts and notifications.

Question 5

Which Fabric component is primarily responsible for routing real-time events to destinations?

A. Warehouse
B. Eventstream
C. Dataflow Gen2
D. Notebook

Answer: B

Explanation:
Eventstream serves as the ingestion and routing layer for streaming architectures.

Question 6

A company requires an end-to-end platform for ingesting, storing, analyzing, and monitoring streaming events.

Which solution should be recommended?

A. Real-Time Intelligence
B. Dataflow Gen2
C. Warehouse
D. SQL Endpoint

Answer: A

Explanation:
Real-Time Intelligence combines ingestion, analytics, monitoring, alerting, and visualization capabilities into a unified platform.

Question 7

Which technology is best suited for analyzing application logs with time-series queries and low-latency reporting?

A. Notebook
B. Warehouse
C. Eventhouse
D. Pipeline

Answer: C

Explanation:
Eventhouse is optimized for streaming analytics, log analysis, and time-series workloads.

Question 8

A solution requires nightly ingestion of source data into a lakehouse.

Which option is most appropriate?

A. Eventstream
B. Data Activator
C. Eventhouse
D. Pipeline

Answer: D

Explanation:
Nightly ingestion is a batch process and is best handled through scheduled pipeline execution.

Question 9

A data engineer needs to continuously enrich streaming events using lookup data and perform custom business-rule calculations.

Which technology should be selected?

A. Spark Structured Streaming
B. Data Activator
C. Eventstream
D. Dashboard

Answer: A

Explanation:
Spark Structured Streaming provides advanced transformation capabilities including joins, aggregations, and custom code execution.

Question 10

Which statement best describes Eventhouse?

A. A workflow orchestration service for ETL processes
B. A low-code data preparation tool
C. A real-time analytics store optimized for event and telemetry data
D. A machine learning training environment

Answer: C

Explanation:
Eventhouse is designed for high-scale event ingestion, real-time analytics, log analytics, and KQL-based querying of streaming data.

Go to the DP-700 Exam Prep Hub main page.

DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Handle duplicate, missing, and late-arriving data (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform batch data
      --> Handle duplicate, missing, and late-arriving data

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

One of the most important responsibilities of a data engineer is ensuring data quality. In real-world data environments, source systems frequently produce data issues that can negatively impact reporting, analytics, machine learning, and business decision-making.

Three of the most common data quality challenges are:

Duplicate data
Missing data
Late-arriving data

Microsoft Fabric provides multiple tools for addressing these challenges, including:

Dataflows Gen2
Data Pipelines
Lakehouses
Data Warehouses
PySpark Notebooks
SQL
Delta Lake tables

For the DP-700 exam, you should understand:

How duplicate data occurs
How to identify and remove duplicates
How to handle missing values
Strategies for managing late-arriving facts and dimensions
Data quality best practices in Fabric
Tools and techniques available across Fabric workloads

Understanding Duplicate Data

Duplicate data occurs when the same logical record appears more than once.

Example:

CustomerID	Name	Email
1001	John Smith	john@email.com
1001	John Smith	john@email.com

Duplicates can occur due to:

Multiple source system loads
Pipeline reruns
Source application errors
Data integration mistakes
Incomplete change tracking

Duplicate records can lead to:

Inflated sales totals
Incorrect customer counts
Poor analytics results
Data integrity issues

Types of Duplicates

Exact Duplicates

Every field is identical.

Example:

OrderID	Amount
5001	250
5001	250

These are usually the easiest to detect.

Partial Duplicates

Records represent the same entity but differ slightly.

Example:

CustomerID	Name
1001	John Smith
1001	Jon Smith

These may require business rules to resolve.

Detecting Duplicate Records Using SQL

A common approach is grouping records and counting occurrences.

Example:

			
SELECT
    CustomerID,
    COUNT(*) AS RecordCount
FROM Customers
GROUP BY CustomerID
HAVING COUNT(*) > 1;

		

This identifies customers appearing multiple times.

Removing Duplicates Using SQL

A common technique uses ROW_NUMBER().

Example:

			
WITH RankedRows AS
(
    SELECT *,
           ROW_NUMBER() OVER
           (
             PARTITION BY CustomerID
             ORDER BY LastModifiedDate DESC
           ) AS RowNum
    FROM Customers
)
SELECT *
FROM RankedRows
WHERE RowNum = 1;

		

This keeps only the most recent record for each customer.

Handling Duplicates Using PySpark

PySpark provides the dropDuplicates() method.

Example:

customers_df = customers_df.dropDuplicates(["CustomerID"])

This removes duplicate records based on CustomerID.

For multiple columns:

			
customers_df = customers_df.dropDuplicates(
    ["CustomerID","Email"]
)

This is frequently used during Lakehouse ingestion processes.

Delta Lake and Duplicate Prevention

Delta Lake tables support MERGE operations that help prevent duplicates.

Example:

			
MERGE INTO Customers AS Target
USING StagingCustomers AS Source
ON Target.CustomerID = Source.CustomerID
WHEN MATCHED THEN
    UPDATE SET *
WHEN NOT MATCHED THEN
    INSERT *

		

Benefits include:

Upserts
Incremental loads
Duplicate prevention
Simplified ETL logic

This is a common Fabric pattern.

Understanding Missing Data

Missing data occurs when expected values are absent.

Example:

CustomerID	Name	Email
1001	John Smith	NULL

Missing values may result from:

Source system limitations
User entry errors
Integration failures
Incomplete records

Why Missing Data Matters

Missing values can cause:

Reporting inaccuracies
Failed transformations
Machine learning issues
Data quality concerns

Example:

Calculating average salary becomes unreliable if many salary values are missing.

Identifying Missing Values

SQL example:

			
SELECT *
FROM Customers
WHERE Email IS NULL;

PySpark example:

			
customers_df.filter(
    customers_df.Email.isNull()
)

Handling Missing Data

Several approaches exist.

Option 1: Replace Missing Values

Example:

			
SELECT
    CustomerID,
    COALESCE(Email,'Unknown')
FROM Customers;

PySpark:

df.fillna("Unknown")

Useful when business rules permit default values.

Option 2: Exclude Records

Example:

			
SELECT *
FROM Customers
WHERE Email IS NOT NULL;

Useful when incomplete records cannot be trusted.

Option 3: Use Business Defaults

Example:

Missing Value	Replacement
Country	Unknown
Discount	0
Status	Pending

Common in dimensional modeling.

Dataflows Gen2 and Missing Data

Dataflows Gen2 provides low-code capabilities for:

Replacing null values
Filtering null rows
Conditional column creation
Data validation

These transformations are frequently used during ingestion and cleansing processes.

Understanding Late-Arriving Data

Late-arriving data refers to records that arrive after the expected processing window.

Example:

An order occurs on January 1st but is not received until January 5th.

Causes of Late-Arriving Data

Common causes include:

Network interruptions
Application outages
Batch processing delays
Replication latency
Manual data entry

Late-arriving data is extremely common in enterprise systems.

Late-Arriving Facts

A late-arriving fact occurs when a transaction arrives after related reporting periods have already been processed.

Example:

OrderDate	Arrival Date
Jan 1	Jan 5

The order belongs to January 1 but arrives later.

Challenges include:

Incorrect historical totals
Incomplete reporting
Fact table inconsistencies

Handling Late-Arriving Facts

Common approaches include:

Reprocessing Historical Periods

Reload affected partitions.

Example:

Reload the previous 7 days during each load.

Benefits:

Simple implementation
Common Fabric approach

Watermark-Based Processing

Track the latest processed timestamp.

Example:

			
WHERE LastModifiedDate >
      @LastProcessedTimestamp

This supports efficient incremental loads.

Sliding Window Loads

Instead of loading only today’s data:

Load the last:

3 days
7 days
30 days

This helps capture delayed records.

Late-Arriving Dimensions

A late-arriving dimension occurs when a fact record arrives before its related dimension record.

Example:

Fact arrives:

CustomerID	Sales
1001	500

Dimension record for CustomerID 1001 has not yet arrived.

Handling Late-Arriving Dimensions

Create an Unknown Member

Dimension table:

CustomerKey	CustomerName
-1	Unknown

Fact records temporarily reference the unknown member.

When the actual dimension arrives:

Update dimension
Correct foreign key relationships

This is a common data warehouse design pattern.

Retry Processing

Pause loading until dimensions arrive.

Useful when:

Data quality requirements are strict
Fact loads depend on dimension availability

Data Quality Monitoring

Fabric solutions should include monitoring for:

Duplicate counts
Null counts
Data freshness
Late-arriving records
Invalid keys

Typical monitoring techniques include:

SQL validation queries
Notebook checks
Pipeline activities
Custom logging tables

Common Fabric Tools for Data Quality

Tool	Use Case
Dataflows Gen2	Data cleansing
Pipelines	Orchestration and validation
PySpark	Large-scale deduplication
SQL	Validation and quality checks
Delta Tables	Upserts and merge operations
Warehouse	Data quality reporting

Best Practices

Prevent Duplicates Early

Validate source data before loading.

Use Incremental Loading

Avoid repeatedly processing entire datasets.

Implement MERGE Operations

Upserts help prevent duplicate records.

Monitor Null Values

Track trends in missing data over time.

Create Unknown Dimension Members

Support late-arriving dimensions.

Use Sliding Window Loads

Capture delayed records efficiently.

Maintain Audit Columns

Examples:

			
CreatedDate
ModifiedDate
LoadDate
SourceSystem
BatchID

		

These assist with troubleshooting and recovery.

DP-700 Exam Tips

Remember these key concepts:

Duplicates can be identified using GROUP BY and COUNT().
ROW_NUMBER() is commonly used for deduplication.
PySpark supports deduplication through dropDuplicates().
Delta Lake MERGE operations support upserts and duplicate prevention.
Missing values can be replaced, filtered, or assigned defaults.
COALESCE() is commonly used in SQL to handle NULL values.
Late-arriving facts often require partition reprocessing or sliding windows.
Late-arriving dimensions are commonly handled using Unknown members.
Incremental loading and watermarking help manage delayed data efficiently.
Data quality monitoring is an essential responsibility of data engineers.

Practice Exam Questions

Question 1

A data engineer wants to identify duplicate customer records in a Fabric Warehouse.

Which SQL technique should be used?

A. GROUP BY with COUNT()

B. ORDER BY

C. DISTINCT

D. UNION

Correct Answer: A

Explanation: GROUP BY combined with COUNT() can identify keys that appear multiple times, making it a common duplicate detection technique.

Question 2

Which PySpark method is specifically designed to remove duplicate records?

A. filter()

B. collect()

C. sort()

D. dropDuplicates()

Correct Answer: D

Explanation: dropDuplicates() removes duplicate records based on one or more specified columns.

Question 3

A data engineer wants to keep only the most recent customer record when duplicates exist.

Which SQL function is most appropriate?

A. SUM()

B. MAX()

C. ROW_NUMBER()

D. COUNT()

Correct Answer: C

Explanation: ROW_NUMBER() allows ranking records within groups so the preferred record can be retained.

Question 4

Which SQL function is commonly used to replace NULL values with a default value?

A. CAST()

B. CONCAT()

C. GROUP BY

D. COALESCE()

Correct Answer: D

Explanation: COALESCE() returns the first non-NULL value and is frequently used to replace missing data.

Question 5

A sales transaction occurred on January 1 but was not received until January 5.

What type of issue is this?

A. Duplicate fact

B. Missing dimension

C. Late-arriving fact

D. Slowly changing dimension

Correct Answer: C

Explanation: The transaction belongs to an earlier business date but arrives after the expected processing window.

Question 6

Which loading strategy helps capture records that arrive several days late?

A. Full reload every hour

B. Sliding window load

C. Static partitioning

D. Manual refresh

Correct Answer: B

Explanation: Sliding window loads reprocess recent periods, allowing delayed records to be captured.

Question 7

A fact record arrives before its related dimension record.

What is this situation called?

A. Incremental loading

B. Duplicate key processing

C. Data drift

D. Late-arriving dimension

Correct Answer: D

Explanation: A late-arriving dimension scenario occurs when fact data is received before the associated dimension data.

Question 8

What is the most common solution for handling late-arriving dimensions in a dimensional model?

A. Delete the fact record

B. Skip the load permanently

C. Create an Unknown member in the dimension table

D. Disable incremental processing

Correct Answer: C

Explanation: An Unknown member allows fact records to load while preserving referential integrity until the dimension arrives.

Question 9

Which Delta Lake operation is commonly used to perform inserts and updates while minimizing duplicate records?

A. DELETE

B. MERGE

C. TRUNCATE

D. CREATE VIEW

Correct Answer: B

Explanation: MERGE supports upsert operations and is widely used for incremental loading and deduplication.

Question 10

Which column is most useful when implementing watermark-based incremental loads?

A. LastModifiedDate

B. Region

C. Category

D. ProductName

Correct Answer: A

Explanation: Watermarking relies on timestamps such as LastModifiedDate to identify new or changed records since the previous load.

Go to the DP-700 Exam Prep Hub main page.

Data Development, DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Group and aggregate data (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Ingest and transform data (30–35%)
   --> Ingest and transform batch data
      --> Group and aggregate data

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Grouping and aggregating data are among the most common and important data transformation operations performed by data engineers. Organizations rarely analyze raw transactional data directly. Instead, they summarize, categorize, and aggregate data to create meaningful business metrics such as total sales, average order value, monthly revenue, customer counts, inventory levels, and operational KPIs.

In Microsoft Fabric, grouping and aggregation can be performed using several technologies, including:

SQL in Fabric Data Warehouses and Lakehouses
PySpark notebooks
Dataflows Gen2
KQL (Kusto Query Language)
Data pipelines as part of larger ETL/ELT processes

For the DP-700 exam, you should understand:

Why grouping and aggregation are important
When to aggregate data
How to implement aggregations using SQL, PySpark, KQL, and Dataflows Gen2
Common aggregation functions
Performance considerations
Aggregations in dimensional modeling and analytics solutions

Why Group and Aggregate Data?

Raw data often contains millions or billions of records.

For example:

OrderID	CustomerID	OrderDate	Amount
1001	C001	2026-01-01	250
1002	C001	2026-01-02	150
1003	C002	2026-01-02	300

Business users usually want summarized information such as:

CustomerID	TotalSales
C001	400
C002	300

This transformation is accomplished through grouping and aggregation.

Benefits include:

Faster analytics
Reduced storage requirements
Easier reporting
Improved dashboard performance
Simplified business intelligence models

Understanding Grouping

Grouping combines records that share common values.

Examples:

Group by:

Customer
Product
Region
Department
Date
Month
Year

Example:

Region	Sales
East	100
East	200
West	300

Grouped by Region:

Region	TotalSales
East	300
West	300

The GROUP BY operation creates logical categories before aggregation calculations occur.

Common Aggregation Functions

Data engineers should be familiar with the most common aggregation functions.

SUM()

Calculates totals.

Example:

			
SELECT Region,
       SUM(SalesAmount) AS TotalSales
FROM Sales
GROUP BY Region;

Result:

Region	TotalSales
East	500000
West	750000

COUNT()

Counts rows.

			
SELECT Region,
       COUNT(*) AS OrderCount
FROM Sales
GROUP BY Region;

Used for:

Number of customers
Number of transactions
Number of products

AVG()

Calculates averages.

			
SELECT ProductCategory,
       AVG(SalesAmount) AS AverageSale
FROM Sales
GROUP BY ProductCategory;

Used for:

Average order value
Average response time
Average inventory level

MIN()

Returns the smallest value.

			
SELECT MIN(OrderDate)
FROM Orders;

Used for:

Earliest order
Lowest temperature
Minimum cost

MAX()

Returns the largest value.

			
SELECT MAX(OrderDate)
FROM Orders;

Used for:

Latest transaction
Highest sales amount
Maximum inventory quantity

Grouping and Aggregation Using SQL

SQL is the most common approach for aggregation in Fabric.

Example:

			
SELECT
    ProductCategory,
    SUM(SalesAmount) AS TotalSales,
    COUNT(*) AS OrderCount,
    AVG(SalesAmount) AS AverageSale
FROM Sales
GROUP BY ProductCategory;

		

This query:

Groups records by category
Calculates total sales
Counts orders
Calculates average sales

Multi-Column Grouping

You can group by multiple columns.

Example:

			
SELECT
    Year,
    Region,
    SUM(SalesAmount) AS TotalSales
FROM Sales
GROUP BY Year, Region;

		

Result:

Year	Region	TotalSales
2025	East	500000
2025	West	700000
2026	East	550000

This provides more granular analysis.

Filtering Aggregated Results with HAVING

WHERE filters rows before aggregation.

HAVING filters after aggregation.

Example:

			
SELECT Region,
       SUM(SalesAmount) AS TotalSales
FROM Sales
GROUP BY Region
HAVING SUM(SalesAmount) > 1000000;

		

Only regions exceeding $1 million in sales are returned.

Aggregation Using PySpark

PySpark is commonly used for large-scale aggregation operations in Lakehouses.

Example:

			
from pyspark.sql.functions import sum
sales_df.groupBy("Region") \
        .agg(sum("SalesAmount").alias("TotalSales")) \
        .show()

Result:

Region	TotalSales
East	500000
West	750000

Multiple Aggregations in PySpark

			
from pyspark.sql.functions import sum, avg, count
sales_df.groupBy("Region").agg(
    sum("SalesAmount").alias("TotalSales"),
    avg("SalesAmount").alias("AvgSales"),
    count("*").alias("OrderCount")
)

		

This performs several calculations in a single operation.

Aggregation Using KQL

KQL is commonly used in Real-Time Intelligence workloads.

Example:

			
Sales
| summarize TotalSales=sum(SalesAmount)
    by Region

Result:

Region	TotalSales
East	500000
West	750000

Multiple Aggregations in KQL

			
Sales
| summarize
    TotalSales=sum(SalesAmount),
    AvgSales=avg(SalesAmount),
    OrderCount=count()
    by Region

		

This pattern is common in real-time analytics.

Aggregation in Dataflows Gen2

Dataflows Gen2 provides a low-code interface.

Using the Group By transformation, users can:

Sum values
Count rows
Calculate averages
Find minimum values
Find maximum values

Typical steps:

Connect to source
Select Group By
Choose grouping columns
Define aggregation functions
Load results

This approach is useful for citizen developers and low-code ETL scenarios.

Aggregation in Dimensional Models

Aggregations are commonly used before loading data into fact and dimension tables.

Example:

Raw transactions:

OrderID	Customer	Amount
1	A	100
2	A	200
3	B	300

Aggregated customer sales:

Customer	TotalSales
A	300
B	300

This summary table can support reporting and dashboards.

Fact Table Aggregations

Fact tables often store:

Transaction-level facts
Daily summaries
Monthly summaries

Examples:

Transaction Fact

OrderID	Amount
1001	50

Daily Aggregate Fact

Date	TotalSales
2026-01-01	50000

Aggregated fact tables improve query performance.

Window Aggregations vs Group Aggregations

Data engineers should understand the difference.

Group Aggregation

Returns one row per group.

			
SELECT Region,
       SUM(SalesAmount)
FROM Sales
GROUP BY Region;

Window Aggregation

Preserves detail rows.

			
SELECT
    OrderID,
    Region,
    SalesAmount,
    SUM(SalesAmount)
        OVER(PARTITION BY Region)
        AS RegionTotal
FROM Sales;

		

Useful for:

Running totals
Rankings
Percentages
Advanced analytics

Performance Considerations

Grouping and aggregation can be expensive.

Best practices include:

Filter Early

Reduce data before aggregation.

WHERE OrderDate >= '2026-01-01'

Aggregate Close to the Source

Avoid moving unnecessary detailed records.

Use Partitioning

Partitioning helps Spark process data efficiently.

Examples:

Date
Region
Customer segment

Use Delta Tables

Delta tables improve performance through:

Data skipping
File optimization
Efficient query execution

Avoid Excessive Cardinality

Grouping on highly unique columns can reduce efficiency.

Bad example:

GROUP BY TransactionID

Good example:

GROUP BY Region

DP-700 Exam Tips

Remember the following:

GROUP BY creates logical groups before aggregation.
HAVING filters aggregated results.
SQL uses GROUP BY.
PySpark uses groupBy() and agg().
KQL uses summarize.
Dataflows Gen2 provides Group By transformations.
Aggregated fact tables improve reporting performance.
Window functions preserve detailed rows while performing calculations.
Aggregations are frequently used when preparing dimensional models.
Filtering before aggregation improves performance.

Practice Exam Questions

Question 1

A data engineer needs to calculate total sales by region in a Fabric Warehouse.

Which SQL function should be used?

A. AVG()

B. COUNT()

C. SUM()

D. MAX()

Correct Answer: C

Explanation: SUM() calculates the total of numeric values. AVG() calculates averages, COUNT() counts rows, and MAX() returns the largest value.

Question 2

A Fabric notebook must calculate the number of orders per customer.

Which aggregation function should be used?

A. COUNT()

B. AVG()

C. MIN()

D. MAX()

Correct Answer: A

Explanation: COUNT() returns the number of rows in each group, making it ideal for counting orders.

Question 3

You need to remove regions with total sales less than $500,000 after aggregation.

Which SQL clause should you use?

A. ORDER BY

B. WHERE

C. DISTINCT

D. HAVING

Correct Answer: D

Explanation: HAVING filters aggregated results after the GROUP BY operation is completed.

Question 4

Which KQL operator is primarily used for aggregation?

A. project

B. summarize

C. extend

D. join

Correct Answer: B

Explanation: The summarize operator performs grouping and aggregation in KQL.

Question 5

A Fabric Dataflow Gen2 developer wants to calculate average sales by product category.

Which transformation should be used?

A. Merge

B. Append

C. Group By

D. Split Column

Correct Answer: C

Explanation: The Group By transformation supports aggregation operations such as averages, sums, counts, minimums, and maximums.

Question 6

What is the primary purpose of a GROUP BY clause?

A. Sort rows

B. Remove duplicates

C. Filter rows

D. Create logical groups for aggregation

Correct Answer: D

Explanation: GROUP BY organizes rows into groups before aggregate calculations are performed.

Question 7

Which PySpark operation performs grouping before aggregation?

A. select()

B. filter()

C. groupBy()

D. orderBy()

Correct Answer: C

Explanation: groupBy() defines the grouping columns that will be used by aggregation functions.

Question 8

Which scenario is most appropriate for a window aggregation?

A. Total sales by region only

B. Average salary by department only

C. Customer counts by state only

D. Display each transaction along with the total sales for its region

Correct Answer: D

Explanation: Window functions preserve detail rows while calculating aggregates across a defined partition.

Question 9

A data engineer groups a dataset by TransactionID, where every TransactionID is unique.

What is the likely result?

A. Improved aggregation performance

B. Reduced cardinality

C. Limited performance benefits because each group contains one row

D. Automatic partition optimization

Correct Answer: C

Explanation: Grouping by a highly unique column creates many groups and often provides little analytical value.

Question 10

When preparing data for a dimensional model, why are aggregated tables often created?

A. To increase data duplication

B. To improve reporting and query performance

C. To eliminate dimension tables

D. To replace fact tables entirely

Correct Answer: B

Explanation: Pre-aggregated tables reduce the amount of data that must be processed during reporting, improving performance and user experience.

Go to the DP-700 Exam Prep Hub main page.

DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Endorse items (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Implement and manage an analytics solution (30–35%)
   --> Configure security and governance
      --> Endorse items

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

As organizations adopt Microsoft Fabric, the number of available data assets can grow rapidly. Data engineers, analysts, business users, and executives may encounter hundreds or even thousands of reports, semantic models, dashboards, warehouses, lakehouses, notebooks, and other data assets.

A common challenge is determining:

Which data assets are trustworthy?
Which reports should be used for executive reporting?
Which semantic models represent official business definitions?
Which datasets have been reviewed and approved?

To address these governance challenges, Microsoft Fabric supports endorsements.

Endorsements help organizations identify trusted and authoritative data assets, making it easier for users to discover and use approved content.

For the DP-700 exam, it is important to understand endorsement types, governance benefits, use cases, and how endorsements differ from security and sensitivity labels.

What Are Endorsements?

An endorsement is a governance feature that allows organizations to identify and promote trusted data assets.

Endorsements help users answer the question:

“Can I trust this data asset?”

Instead of searching through numerous reports and datasets, users can quickly identify endorsed items that have been reviewed and approved.

Purpose of Endorsements

Organizations use endorsements to:

Improve data discoverability
Promote trusted assets
Reduce duplicate reports
Encourage consistent reporting
Improve governance
Increase user confidence
Establish authoritative data sources

Endorsement Types

Microsoft Fabric supports two primary endorsement levels:

Promoted

Certified

These endorsement levels indicate different degrees of trust and governance.

Promoted Items

A Promoted item indicates:

The content creator believes the item is valuable.
The item is recommended for broader use.
The item may not have gone through formal governance review.

Think of Promoted as:

Recommended Content

Examples:

Frequently used reports
Department dashboards
Common semantic models
Team-approved datasets

Characteristics of Promoted Items

Promoted items:

Are easier to discover
Indicate useful content
Can be designated by authorized users
Do not necessarily represent official organizational standards

Example

A Sales team creates a dashboard used by dozens of users.

The dashboard is reliable and widely used.

The owner marks it as:

Promoted

This helps users identify it as recommended content.

Certified Items

Certified is a higher endorsement level.

Certified items have typically undergone formal review and approval processes.

Think of Certified as:

Official Trusted Content

Examples:

Executive reporting datasets
Enterprise semantic models
Corporate KPI reports
Official financial dashboards

Characteristics of Certified Items

Certified items:

Represent authoritative data
Follow governance standards
Have undergone validation
Are approved by designated governance teams
Should be used whenever possible

Example

A Finance semantic model contains:

Revenue
Expenses
Profit
Corporate KPIs

The governance team validates the model and certifies it.

The model becomes:

Certified

Users now know it represents official business definitions.

Comparing Promoted and Certified

Feature	Promoted	Certified
Recommended by creator	Yes	Yes
Formal review required	No	Yes
Governance approval	Optional	Required
Official organizational source	Not necessarily	Yes
Highest trust level	No	Yes

Why Endorsements Matter

Without endorsements:

			
Sales Report V1
Sales Report V2
Sales Report Final
Sales Report Final2
Sales Dashboard New

		

Users may not know which asset to trust.

With endorsements:

			
Sales Dashboard
(Certified)

The preferred asset becomes obvious.

Supported Fabric Items

Endorsements can be applied to many Fabric assets, including:

Semantic Models
Reports
Dashboards
Data Warehouses
Lakehouses
Dataflows
Other supported Fabric artifacts

Supported item types may evolve as Microsoft Fabric continues to expand.

Endorsements and Data Discovery

One major benefit of endorsements is improved discoverability.

Users searching for assets can identify:

Promoted content
Certified content

This reduces confusion and encourages reuse of trusted assets.

Governance Benefits

Endorsements support governance initiatives by helping organizations:

Establish trusted data sources
Reduce shadow analytics
Minimize duplicate content
Improve reporting consistency
Promote enterprise standards

Endorsements vs Security Permissions

A common DP-700 exam topic is distinguishing endorsements from security.

Endorsements	Permissions
Identify trusted content	Control access
Governance feature	Security feature
Improve discoverability	Restrict usage
Indicate quality	Grant authorization

Example:

A report may be:

Certified

But users still require permissions to access it.

Certification does not grant access.

Endorsements vs Sensitivity Labels

Another frequently tested distinction.

Endorsements	Sensitivity Labels
Indicate trustworthiness	Indicate sensitivity
Governance and quality	Classification and protection
Help users find trusted content	Help users identify sensitive content

Example:

			
Certified Report
Highly Confidential

Both labels may exist simultaneously.

The report is:

Trusted (Certified)
Sensitive (Highly Confidential)

Endorsements vs Data Lineage

Endorsements	Data Lineage
Indicates trust	Shows data flow
Governance tool	Dependency tracking tool

Data lineage answers:

Where did this data come from?

Endorsements answer:

Can I trust this asset?

Common DP-700 Exam Scenarios

Scenario 1

Requirement:

Users need to identify official KPI definitions.

Solution:

Use Certified semantic models.

Scenario 2

Requirement:

A department wants to recommend a dashboard without formal review.

Solution:

Use Promoted endorsement.

Scenario 3

Requirement:

An executive dashboard has been validated by the governance team.

Solution:

Apply Certified endorsement.

Scenario 4

Requirement:

A report contains highly sensitive financial information.

Solution:

Apply a sensitivity label.

Not an endorsement.

Endorsement Workflow

A common governance workflow:

			
Create Asset
      ↓
Validate Asset
      ↓
Promote Asset
      ↓
Governance Review
      ↓
Certify Asset

		

This process improves trust and consistency.

Best Practices

Certify Enterprise Assets

Certify:

Corporate KPI datasets
Financial reports
Enterprise semantic models

Promote Useful Content

Promote:

Department dashboards
Frequently used reports
Shared analytics assets

Establish Governance Processes

Define:

Who can certify content
Review procedures
Approval standards

Avoid Certifying Everything

Certification should remain meaningful and reserved for truly authoritative assets.

Combine Governance Features

Use endorsements alongside:

Sensitivity labels
Lineage tracking
Security permissions
Data cataloging

DP-700 Exam Focus Areas

You should understand:

✓ Purpose of endorsements

✓ Promoted endorsements

✓ Certified endorsements

✓ Governance benefits

✓ Data discovery improvements

✓ Trusted data sources

✓ Promoted versus Certified

✓ Endorsements versus permissions

✓ Endorsements versus sensitivity labels

✓ Endorsements versus lineage

✓ Common governance scenarios

Practice Exam Questions

Question 1

What is the primary purpose of endorsements in Microsoft Fabric?

A. Encrypt sensitive data

B. Identify trusted and recommended data assets

C. Filter rows of data

D. Control workspace permissions

Answer: B

Explanation

Endorsements help users identify trusted, recommended, and authoritative data assets within Fabric.

Question 2

Which endorsement level represents the highest level of organizational trust?

A. Endorsed

B. Promoted

C. Confidential

D. Certified

Answer: D

Explanation

Certified is the highest endorsement level and indicates formal governance review and approval.

Question 3

A department wants to highlight a useful dashboard without requiring formal governance approval.

Which endorsement should be used?

A. Certified

B. Promoted

C. Confidential

D. Restricted

Answer: B

Explanation

Promoted endorsements indicate recommended content without requiring formal certification processes.

Question 4

What is a key characteristic of a Certified item?

A. It automatically grants workspace access.

B. It is encrypted.

C. It automatically receives a sensitivity label.

D. It has undergone formal validation and approval.

Answer: D

Explanation

Certified items have been reviewed and approved according to organizational governance standards.

Question 5

How do endorsements differ from security permissions?

A. Endorsements classify sensitivity levels.

B. Endorsements indicate trustworthiness, while permissions control access.

C. Endorsements encrypt content.

D. Endorsements implement Row-Level Security.

Answer: B

Explanation

Permissions determine who can access an asset, while endorsements indicate whether the asset is trusted.

Question 6

Which statement about Promoted items is correct?

A. They require formal governance certification.

B. They cannot be used by business users.

C. They indicate content that is recommended for broader use.

D. They automatically become Certified after publication.

Answer: C

Explanation

Promoted items highlight useful and recommended content without formal certification requirements.

Question 7

A governance team reviews and approves an enterprise semantic model that contains official KPI definitions.

Which endorsement should be applied?

A. Public

B. Promoted

C. Internal

D. Certified

Answer: D

Explanation

Certified endorsement is appropriate for formally reviewed and approved enterprise assets.

Question 8

What problem do endorsements primarily help solve?

A. Unauthorized access

B. Data encryption

C. User identification

D. Difficulty identifying trusted content

Answer: D

Explanation

Endorsements help users distinguish trusted assets from numerous available reports and datasets.

Question 9

A report is marked as Certified.

What does this indicate?

A. It is an authoritative and approved data asset.

B. It is automatically encrypted.

C. It is accessible to all users.

D. It contains confidential information.

Answer: A

Explanation

Certification indicates that the asset has been validated and approved as a trusted source.

Question 10

Which statement best describes the relationship between endorsements and sensitivity labels?

A. They are identical governance features.

B. Sensitivity labels replace endorsements.

C. Endorsements indicate trustworthiness, while sensitivity labels indicate data sensitivity.

D. Certified items cannot have sensitivity labels.

Answer: C

Explanation

Endorsements focus on trust and quality, while sensitivity labels focus on classification and protection requirements.

Exam Tip

One of the most common DP-700 exam traps is confusing endorsements, sensitivity labels, and security permissions.

Remember:

Requirement	Solution
Identify trusted content	Endorsements
Classify sensitive data	Sensitivity Labels
Control who can access data	Permissions
Track data origins	Lineage

A useful memory aid is:

Promoted = Recommended
Certified = Official
Sensitivity Label = Sensitive
Permission = Access

If the exam question focuses on helping users identify the most trustworthy or authoritative asset, the correct answer is often Promoted or Certified endorsement, not a security control.

Go to the DP-700 Exam Prep Hub main page.

Data Security, DP-700, Microsoft Certification, Microsoft Fabric June 3, 2026

Apply sensitivity labels to items (DP-700 Exam Prep)

This post is a part of the DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric Exam Prep Hub.
This topic falls under these sections:
Implement and manage an analytics solution (30–35%)
   --> Configure security and governance
      --> Apply sensitivity labels to items

Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available from the hub's main page below the exam topics section.

Introduction

Data is one of an organization’s most valuable assets, and not all data carries the same level of sensitivity. Some information can be shared broadly across the organization, while other information must be protected due to regulatory, legal, contractual, or business requirements.

Examples include:

Financial reports
Employee records
Customer information
Healthcare data
Intellectual property
Confidential business plans

Microsoft Fabric integrates with Microsoft Information Protection (MIP) and Microsoft Purview sensitivity labels, enabling organizations to classify and protect data assets throughout their lifecycle.

Sensitivity labels help users understand the importance of data, enforce governance policies, and support compliance initiatives. They can be applied to many Fabric items, including reports, semantic models, dashboards, and other assets.

For the DP-700 exam, it is important to understand what sensitivity labels are, how they work, how they are applied, and how they differ from other security mechanisms such as Row-Level Security (RLS), Object-Level Security (OLS), and workspace permissions.

What Are Sensitivity Labels?

A sensitivity label is a classification tag that identifies the sensitivity level of data or content.

Examples include:

Label	Typical Meaning
Public	Safe for general sharing
General	Internal business use
Confidential	Restricted business information
Highly Confidential	Highly sensitive information
Restricted	Maximum protection required

Organizations can create custom sensitivity labels to align with their governance policies.

Purpose of Sensitivity Labels

Sensitivity labels help organizations:

Classify data consistently
Protect sensitive information
Improve data governance
Support regulatory compliance
Enable data discovery
Increase user awareness
Reduce accidental data exposure

Sensitivity labels serve as both a classification mechanism and, in some scenarios, a protection mechanism.

Sensitivity Labels in Microsoft Fabric

Microsoft Fabric integrates with Microsoft Purview Information Protection.

This integration allows labels to be applied to Fabric assets and propagated through data workflows.

Examples of Fabric items that can support sensitivity labels include:

Reports
Semantic Models
Dashboards
Data Warehouses
Lakehouses
Notebooks
Dataflows
Other supported Fabric artifacts

How Sensitivity Labels Work

A sensitivity label is associated with an item.

Example:

			
Executive Financial Report
Label: Highly Confidential

Users accessing the item can immediately identify its classification level.

The label travels with the item and may also propagate to downstream artifacts depending on organizational policies and supported scenarios.

Common Sensitivity Label Classifications

Public

Data intended for unrestricted access.

Examples:

Public website content
Marketing brochures
Published documentation

General

Data intended for internal use.

Examples:

Departmental reports
Internal project tracking

Confidential

Data requiring controlled access.

Examples:

Financial reports
Customer information
Internal analytics

Highly Confidential

Data requiring strict protection.

Examples:

Payroll information
Acquisition plans
Executive strategy documents

Restricted

Data requiring maximum protection.

Examples:

Legal investigations
Security credentials
Highly regulated information

Applying Sensitivity Labels

Sensitivity labels can be applied manually or automatically depending on organizational configurations.

Manual Labeling

Users select the appropriate label.

Example:

			
Report
 ↓
Apply Label
 ↓
Confidential

		

Manual labeling is commonly used when users understand the business context of the data.

Automatic Labeling

Organizations may configure policies that automatically apply labels based on:

Sensitive information types
Data patterns
Business rules
Compliance requirements

Example:

			
Contains Credit Card Data
           ↓
Apply Highly Confidential

Label Inheritance and Propagation

One of the most important DP-700 exam topics related to sensitivity labels is inheritance.

Labels may propagate from source items to downstream artifacts.

Example:

			
Lakehouse
(Confidential)
      ↓
Semantic Model
      ↓
Report

		

The downstream item may inherit the sensitivity label from its source.

This helps maintain governance consistency throughout the analytics lifecycle.

Benefits of Label Propagation

Without propagation:

			
Sensitive Data
       ↓
Unlabeled Report

Risk:

Users may unknowingly share sensitive information.

With propagation:

			
Sensitive Data
       ↓
Confidential Report

Users are immediately aware of the sensitivity level.

Sensitivity Labels vs Security Permissions

This distinction is frequently tested on certification exams.

Sensitivity Labels	Security Permissions
Classify data	Control access
Provide governance context	Enforce authorization
Improve awareness	Restrict usage
Support compliance	Protect resources

Example:

A report may be labeled:

Highly Confidential

But unless appropriate permissions exist, the label alone does not automatically prevent access.

Sensitivity Labels vs Row-Level Security

Sensitivity Labels	Row-Level Security
Classify content	Filter data rows
Governance feature	Access control feature
Applies to items	Applies to data records

Example:

RLS:

			
East Manager
↓
East Region Rows Only

Sensitivity Label:

Confidential Report

Both can be used together.

Sensitivity Labels vs Dynamic Data Masking

Sensitivity Labels	Dynamic Data Masking
Classifies data	Obscures sensitive values
Governance-focused	Security-focused
Does not change data display	Changes displayed values

Example:

Label:

Highly Confidential

Masking:

XXXX-XXXX-1234

Sensitivity Labels and Compliance

Sensitivity labels play an important role in compliance initiatives such as:

GDPR
HIPAA
PCI DSS
SOX
Internal governance programs

They help organizations:

Identify sensitive assets
Demonstrate governance controls
Improve audit readiness

Sensitivity Labels and Microsoft Purview

Microsoft Purview provides centralized governance capabilities.

Organizations can use Purview to:

Define sensitivity labels
Publish labels
Manage classification policies
Track protected content
Support compliance reporting

Fabric integrates with these governance capabilities.

Real-World Scenarios

Scenario 1

A finance report contains quarterly earnings information.

Solution:

Apply a Confidential sensitivity label.

Scenario 2

A payroll dataset contains salary and compensation data.

Solution:

Apply a Highly Confidential sensitivity label.

Scenario 3

A public product catalog is intended for external customers.

Solution:

Apply a Public label.

Scenario 4

A report inherits data from a Confidential semantic model.

Result:

The report may inherit the Confidential label through label propagation.

Best Practices

Establish a Clear Classification Framework

Create standardized labels such as:

Public
General
Confidential
Highly Confidential

Use Consistent Labeling

Apply labels consistently across Fabric assets.

Leverage Label Propagation

Allow downstream artifacts to inherit labels when appropriate.

Train Users

Ensure users understand:

Label meanings
Sharing responsibilities
Governance requirements

Combine Labels with Security Controls

Use labels alongside:

Workspace permissions
Item permissions
Row-Level Security
Object-Level Security
Dynamic Data Masking

Review Labels Regularly

Data classifications may change over time.

DP-700 Exam Focus Areas

You should understand:

✓ Sensitivity label concepts

✓ Microsoft Purview integration

✓ Classification levels

✓ Manual labeling

✓ Automatic labeling

✓ Label inheritance

✓ Label propagation

✓ Governance benefits

✓ Compliance scenarios

✓ Sensitivity labels versus security permissions

✓ Sensitivity labels versus RLS

✓ Sensitivity labels versus Dynamic Data Masking

Practice Exam Questions

Question 1

What is the primary purpose of a sensitivity label in Microsoft Fabric?

A. To classify and identify the sensitivity level of data

B. To encrypt data at rest

C. To filter rows of data

D. To assign workspace roles

Answer: A

Explanation

Sensitivity labels classify data according to its sensitivity and governance requirements. They are primarily used for data classification and protection awareness.

Question 2

Which Microsoft service provides the sensitivity labeling framework used by Microsoft Fabric?

A. Microsoft Purview

B. Microsoft Defender

C. Microsoft Sentinel

D. Azure Key Vault

Answer: A

Explanation

Microsoft Fabric integrates with Microsoft Purview Information Protection to provide sensitivity labeling capabilities.

Question 3

A company wants reports containing payroll information to be clearly identified as highly sensitive.

Which feature should be used?

A. Dynamic Data Masking

B. Row-Level Security

C. Sensitivity Labels

D. Deployment Pipelines

Answer: C

Explanation

Sensitivity labels classify and identify the sensitivity level of data assets such as payroll reports.

Question 4

What is label propagation?

A. Automatic workspace creation

B. Automatic dataset refresh

C. Automatic role assignment

D. Automatic inheritance of sensitivity labels to downstream items

Answer: D

Explanation

Label propagation helps maintain consistent governance by carrying sensitivity classifications to derived artifacts.

Question 5

Which statement best describes the relationship between sensitivity labels and security permissions?

A. Sensitivity labels replace security permissions.

B. Security permissions automatically create labels.

C. Sensitivity labels classify data, while permissions control access.

D. Sensitivity labels filter rows of data.

Answer: C

Explanation

Labels provide classification and governance context, while permissions determine who can access resources.

Question 6

A report inherits data from a semantic model labeled Confidential.

What may happen if label propagation is enabled?

A. The report may inherit the Confidential label.

B. The report becomes encrypted automatically.

C. The report is deleted after publication.

D. Workspace permissions are removed.

Answer: A

Explanation

Label propagation can automatically apply inherited classifications to downstream assets.

Question 7

Which classification would generally represent the highest level of protection?

A. Public

B. General

C. Confidential

D. Highly Confidential

Answer: D

Explanation

Highly Confidential labels are typically used for the most sensitive business information.

Question 8

Which statement about sensitivity labels is correct?

A. They filter records based on user identity.

B. They hide columns from users.

C. They help classify and govern data assets.

D. They assign workspace roles.

Answer: C

Explanation

Sensitivity labels primarily support classification, governance, and compliance initiatives.

Question 9

A company wants to automatically classify files containing credit card information.

Which capability supports this requirement?

A. Automatic sensitivity labeling

B. Dynamic Data Masking

C. Workspace Viewer permissions

D. Object-Level Security

Answer: A

Explanation

Automatic labeling policies can identify sensitive information patterns and apply appropriate labels.

Question 10

Why should sensitivity labels be combined with security controls?

A. Labels automatically replace encryption.

B. Labels alone do not control access to data.

C. Labels remove the need for governance policies.

D. Labels prevent all data leakage scenarios.

Answer: B

Explanation

Sensitivity labels provide classification and governance information, but access controls such as permissions, RLS, and masking are still required to secure data.

Exam Tip

A common DP-700 exam challenge is distinguishing classification technologies from access control technologies.

Remember:

Requirement	Solution
Classify data sensitivity	Sensitivity Labels
Restrict who can access data	Permissions
Restrict which rows users see	Row-Level Security
Hide sensitive values	Dynamic Data Masking
Hide tables or objects	Object-Level Security

If the question focuses on identifying, classifying, labeling, governing, or tracking sensitive data, the correct answer is often Sensitivity Labels rather than a traditional security control.

Go to the DP-700 Exam Prep Hub main page.

Exam Prep Hubs available on The Data Community

Exam Prep Hub for AI-900: Microsoft Azure AI Fundamentals

Exam Prep Hub for PL-300: Microsoft Power BI Data Analyst

Exam Prep Hub for DP-600: Implementing Analytics Solutions Using Microsoft Fabric

Introduction

What Is KQL?

Why Use KQL for Streaming Data?

KQL in Microsoft Fabric

Eventhouses

KQL Databases

KQL Processing Workflow

Understanding KQL Query Structure

Filtering Streaming Data

Selecting Columns

Sorting Results

Aggregating Data with Summarize

Grouping Data

Time-Based Analysis

Working with Time-Series Data

Parsing Semi-Structured Data

Creating Calculated Columns

Joining Streaming Data

Materialized Views

Update Policies

Streaming Ingestion

KQL vs Spark Structured Streaming

Use KQL When:

Use Spark When:

KQL vs T-SQL

Performance Best Practices

Filter Early

Project Only Required Columns

Use Materialized Views

Use Appropriate Time Bins

Common DP-700 Exam Scenarios

Scenario 1

Scenario 2

Scenario 3

Scenario 4

DP-700 Exam Tips

Practice Exam Questions

Question 1

Question 2

Question 3

Question 4

Question 5

Question 6

Question 7

Question 8

Question 9

Question 10

Introduction

What Is Spark Structured Streaming?

How Structured Streaming Works

Common Sources

Common Destinations

Structured Streaming in Microsoft Fabric

Structured Streaming vs Batch Processing

Use Batch Processing When:

Use Structured Streaming When:

Micro-Batch Processing

Reading Streaming Data

Writing Streaming Data

Delta Lake Integration

Checkpointing

Triggers

Stateful vs Stateless Processing

Stateless Processing

Stateful Processing

Stream Aggregations

Common Streaming Scenarios in Fabric

IoT Monitoring

Application Telemetry

Real-Time Business Analytics

Structured Streaming vs Eventstreams

Production Best Practices

Use Spark Job Definitions

Configure Retry Policies

Always Use Checkpoints

Optimize Partitioning