This post is a part of the AI-901: Microsoft Azure AI Fundamentals Exam Prep Hub.
This topic falls under these sections:
Identify AI concepts and capabilities (40–45%)
--> Describe principles of responsible AI
--> Describe considerations for privacy and security in an AI Solution
Note that there are 10 practice questions (with answers and explanations) for each section to help you solidify your knowledge of the material. Also, there are 2 practice tests with 60 questions each available on the hub below the exam topics section.
Privacy and security are essential principles of Responsible AI and important topics for the AI-901 certification exam. Microsoft emphasizes that AI systems must protect sensitive information, respect user privacy, and defend against unauthorized access or malicious attacks.
As AI systems increasingly process personal, financial, medical, and business data, organizations must ensure that their AI solutions are secure and trustworthy.
What Are Privacy and Security in AI?
Although related, privacy and security are different concepts.
| Concept | Meaning |
|---|---|
| Privacy | Protecting personal and sensitive information and ensuring proper data usage |
| Security | Protecting systems, models, and data from unauthorized access, attacks, or misuse |
Both principles are critical when developing and deploying AI systems.
Why Privacy and Security Matter
AI systems often process large amounts of sensitive information, including:
- Personal data
- Financial records
- Medical information
- Images and videos
- Voice recordings
- Customer behavior data
- Business intelligence data
If privacy or security is compromised, organizations may face:
- Data breaches
- Identity theft
- Financial loss
- Legal penalties
- Loss of customer trust
- Regulatory violations
Responsible AI requires organizations to safeguard both the data and the systems that use it.
Privacy Considerations in AI
Collect Only Necessary Data
Organizations should collect only the data required for the AI solution to function properly.
This concept is often called data minimization.
Example
A movie recommendation system may need viewing preferences but may not need a user’s medical history.
Collecting unnecessary data increases privacy risks.
User Consent and Transparency
Users should understand:
- What data is being collected
- Why the data is being collected
- How the data will be used
- Who can access the data
Organizations should obtain appropriate user consent before collecting or processing personal information.
Example
A voice assistant application should clearly inform users that voice recordings are being stored and analyzed.
Protect Sensitive Information
Sensitive data should be carefully protected during:
- Collection
- Storage
- Processing
- Transmission
Examples of sensitive information include:
- Social Security numbers
- Credit card data
- Medical records
- Biometric data
Organizations often use encryption and access controls to protect sensitive data.
Anonymization and Masking
Organizations can reduce privacy risks by removing or hiding personally identifiable information (PII).
Techniques include:
- Anonymization
- Data masking
- Tokenization
Example
A healthcare AI system may replace patient names with anonymous identifiers before training a model.
Compliance with Regulations
Organizations must comply with privacy laws and regulations.
Examples include:
- GDPR (General Data Protection Regulation)
- HIPAA (Health Insurance Portability and Accountability Act)
- CCPA (California Consumer Privacy Act)
AI systems should be designed with regulatory compliance in mind.
Security Considerations in AI
Protecting AI Systems from Unauthorized Access
AI systems should include strong authentication and authorization controls.
Examples
- Multi-factor authentication (MFA)
- Role-based access control (RBAC)
- Identity management systems
Only authorized users should be able to access sensitive models or data.
Securing Data
Data should be protected both:
- At rest (stored data)
- In transit (moving across networks)
Encryption is commonly used to secure data in both situations.
Protecting Models from Attacks
AI systems can be targets for malicious attacks.
Examples include:
- Adversarial attacks
- Data poisoning
- Model theft
- Prompt injection attacks in generative AI systems
Organizations should monitor for suspicious activity and secure AI infrastructure.
Adversarial Attacks
An adversarial attack occurs when someone intentionally manipulates input data to fool an AI model.
Example
Small changes to an image may cause an AI vision system to incorrectly identify an object.
These attacks can reduce reliability and create safety risks.
Data Poisoning
Data poisoning occurs when attackers intentionally insert misleading or malicious data into training datasets.
Example
An attacker adds fraudulent examples into a spam detection dataset so spam messages are classified as safe.
This can compromise model accuracy and trustworthiness.
Generative AI Security Risks
Generative AI introduces additional privacy and security challenges.
Examples include:
- Prompt injection attacks
- Exposure of confidential data
- Harmful content generation
- Leakage of sensitive training data
Organizations should implement safeguards such as:
- Content filtering
- Access restrictions
- Human review
- Monitoring and logging
Shared Responsibility in Cloud AI
When using cloud-based AI services such as Microsoft Azure AI Services, security responsibilities are shared.
| Microsoft Responsibilities | Customer Responsibilities |
|---|---|
| Physical infrastructure security | User access management |
| Network security | Proper configuration |
| Cloud platform protection | Data governance |
| Service availability | Compliance and policy management |
Understanding the shared responsibility model is important for cloud security.
Real-World Example
Scenario: AI Banking Chatbot
A bank deploys an AI chatbot that helps customers manage accounts.
Privacy Considerations
- Protect customer financial data
- Obtain consent for data collection
- Limit access to sensitive records
- Mask account numbers in logs
Security Considerations
- Use encryption
- Require authentication
- Prevent unauthorized access
- Monitor for suspicious activity
- Protect against prompt injection attacks
Risk Mitigation Strategies
- Access controls
- Security monitoring
- Data anonymization
- Regular audits
- Employee security training
This type of scenario aligns well with AI-901 exam questions.
Privacy vs. Security
A common exam concept is understanding the difference between privacy and security.
Privacy Focuses On:
- Proper use of personal data
- User consent
- Data collection practices
- Data sharing limitations
Security Focuses On:
- Protecting systems and data
- Preventing attacks
- Access control
- Encryption
- Threat detection
Privacy and security work together but are not the same thing.
Microsoft Responsible AI Principles
Microsoft identifies privacy and security as one of six core Responsible AI principles:
- Fairness
- Reliability and safety
- Privacy and security
- Inclusiveness
- Transparency
- Accountability
For AI-901, understand that privacy and security focus on protecting both users and AI systems.
Best Practices for Privacy and Security in AI
Organizations commonly use the following practices:
Encryption
Protect data by encrypting it:
- At rest
- In transit
Access Controls
Restrict system access using:
- RBAC
- MFA
- Identity management
Data Governance
Establish policies for:
- Data handling
- Data retention
- Data sharing
- Compliance
Monitoring and Logging
Track suspicious behavior and system activity to detect threats early.
Regular Security Testing
Perform:
- Vulnerability scans
- Penetration testing
- Security reviews
Human Oversight
Humans should monitor high-risk AI systems and review sensitive outputs.
Important AI-901 Exam Tips
For the exam, remember these key points:
- Privacy protects personal and sensitive information.
- Security protects systems, models, and data from attacks or unauthorized access.
- Data minimization reduces privacy risk.
- Encryption protects data at rest and in transit.
- AI systems can face adversarial attacks and data poisoning.
- Generative AI introduces additional security concerns.
- User consent and transparency are important privacy considerations.
- Privacy and security are one of Microsoft’s six Responsible AI principles.
Quick Knowledge Check
Question 1
What is the difference between privacy and security?
Answer
Privacy focuses on proper handling of personal data, while security focuses on protecting systems and data from threats and unauthorized access.
Question 2
What is data minimization?
Answer
Collecting only the data necessary for an AI solution to function.
Question 3
What is an adversarial attack?
Answer
An attempt to intentionally manipulate AI inputs to fool the model into producing incorrect results.
Question 4
Why is encryption important in AI systems?
Answer
It helps protect sensitive data from unauthorized access during storage and transmission.
Practice Exam Questions
Question 1
A company develops an AI-powered healthcare application that stores patient medical records.
Which practice BEST helps protect sensitive patient data?
A. Publicly sharing all training data
B. Encrypting stored and transmitted data
C. Removing all authentication requirements
D. Allowing unrestricted administrator access
Correct Answer
B. Encrypting stored and transmitted data
Explanation
Encryption protects sensitive information both while stored (at rest) and while moving across networks (in transit). This is a key privacy and security practice for AI systems handling confidential data.
Why the Other Answers Are Incorrect
A. Publicly sharing all training data
This would create major privacy risks.
C. Removing all authentication requirements
Authentication is necessary for security.
D. Allowing unrestricted administrator access
Access should be limited and controlled.
Question 2
What is the PRIMARY focus of privacy in an AI solution?
A. Preventing hardware failures
B. Protecting personal and sensitive information
C. Increasing processing speed
D. Improving graphics performance
Correct Answer
B. Protecting personal and sensitive information
Explanation
Privacy focuses on ensuring personal data is collected, stored, shared, and used responsibly and lawfully.
Why the Other Answers Are Incorrect
A. Preventing hardware failures
This relates to infrastructure reliability.
C. Increasing processing speed
Performance optimization is unrelated to privacy.
D. Improving graphics performance
Graphics performance is unrelated to Responsible AI privacy principles.
Question 3
Which scenario BEST demonstrates data minimization?
A. Collecting all available user data regardless of need
B. Collecting only the information necessary for the AI solution to function
C. Sharing customer data with external organizations
D. Storing user data indefinitely
Correct Answer
B. Collecting only the information necessary for the AI solution to function
Explanation
Data minimization means limiting data collection to only what is necessary for a specific purpose, reducing privacy risks.
Why the Other Answers Are Incorrect
A. Collecting all available user data regardless of need
This increases privacy risk.
C. Sharing customer data with external organizations
This may create additional privacy concerns.
D. Storing user data indefinitely
Long-term storage may increase compliance and security risks.
Question 4
An attacker slightly modifies an image so that an AI vision system incorrectly identifies an object.
What type of attack is this?
A. Data normalization
B. Adversarial attack
C. Batch processing
D. Role-based access control
Correct Answer
B. Adversarial attack
Explanation
Adversarial attacks intentionally manipulate inputs to fool AI systems into making incorrect predictions or classifications.
Why the Other Answers Are Incorrect
A. Data normalization
Normalization prepares data for analysis.
C. Batch processing
Batch processing refers to grouped data operations.
D. Role-based access control
RBAC is a security access management method.
Question 5
Which security measure helps ensure only authorized users can access an AI system?
A. Increasing training data size
B. Role-based access control (RBAC)
C. Removing encryption
D. Disabling audit logs
Correct Answer
B. Role-based access control (RBAC)
Explanation
RBAC restricts access based on user roles and permissions, helping secure AI systems and sensitive data.
Why the Other Answers Are Incorrect
A. Increasing training data size
Training data size does not control access.
C. Removing encryption
Removing encryption weakens security.
D. Disabling audit logs
Audit logs help monitor and investigate security events.
Question 6
What is the PRIMARY purpose of encryption in AI systems?
A. To increase model accuracy
B. To protect data from unauthorized access
C. To reduce cloud costs
D. To eliminate the need for passwords
Correct Answer
B. To protect data from unauthorized access
Explanation
Encryption converts data into a protected format that unauthorized users cannot easily read.
It is commonly used to secure sensitive information.
Why the Other Answers Are Incorrect
A. To increase model accuracy
Encryption does not improve prediction quality.
C. To reduce cloud costs
Encryption is a security measure, not a cost optimization tool.
D. To eliminate the need for passwords
Authentication may still be required.
Question 7
A company clearly informs users about what personal information is being collected and how it will be used before collecting the data.
What privacy concept does this BEST represent?
A. User consent and transparency
B. Adversarial testing
C. Model drift
D. Data poisoning
Correct Answer
A. User consent and transparency
Explanation
Responsible AI systems should inform users about data collection practices and obtain appropriate consent before using personal data.
Why the Other Answers Are Incorrect
B. Adversarial testing
Adversarial testing evaluates resistance to attacks.
C. Model drift
Model drift refers to performance changes over time.
D. Data poisoning
Data poisoning involves malicious manipulation of training data.
Question 8
An attacker intentionally inserts misleading examples into a training dataset to reduce model accuracy.
What is this called?
A. Encryption
B. Data masking
C. Data poisoning
D. Data normalization
Correct Answer
C. Data poisoning
Explanation
Data poisoning occurs when attackers deliberately manipulate training data to negatively affect AI model behavior.
Why the Other Answers Are Incorrect
A. Encryption
Encryption protects data confidentiality.
B. Data masking
Data masking hides sensitive information.
D. Data normalization
Normalization standardizes data values.
Question 9
Which statement BEST describes the difference between privacy and security?
A. Privacy and security are identical concepts
B. Privacy focuses on proper data usage, while security focuses on protecting systems and data from threats
C. Privacy focuses only on hardware devices
D. Security applies only to cloud computing
Correct Answer
B. Privacy focuses on proper data usage, while security focuses on protecting systems and data from threats
Explanation
Privacy concerns how personal data is collected and used, while security focuses on preventing unauthorized access, attacks, and data breaches.
Why the Other Answers Are Incorrect
A. Privacy and security are identical concepts
They are related but distinct principles.
C. Privacy focuses only on hardware devices
Privacy primarily concerns information handling.
D. Security applies only to cloud computing
Security applies to all computing environments.
Question 10
Which Microsoft Responsible AI principle focuses on protecting sensitive information and securing AI systems?
A. Fairness
B. Inclusiveness
C. Privacy and security
D. Transparency
Correct Answer
C. Privacy and security
Explanation
The Privacy and Security principle focuses on safeguarding personal data and protecting AI systems from threats, misuse, and unauthorized access.
Why the Other Answers Are Incorrect
A. Fairness
Fairness focuses on avoiding unjust bias and discrimination.
B. Inclusiveness
Inclusiveness focuses on designing systems accessible to diverse users.
D. Transparency
Transparency focuses on explainability and understanding AI decisions.
Final Thoughts
Privacy and security are foundational Responsible AI principles and key topics for the AI-901 certification exam. Microsoft expects candidates to understand how AI systems handle sensitive data, how security threats can affect AI solutions, and how organizations can protect both users and systems.
Strong privacy and security practices help organizations build trustworthy AI solutions while reducing legal, operational, and reputational risks.
Go to the AI-901 Exam Prep Hub main page
The Data Community 