This post is a part of the AI-200: Developing AI Cloud Solutions on Azure Exam Prep Hub.
This topic falls under these sections:
Develop AI solutions by using Azure data management services (25–30%)
--> Develop AI solutions by using Azure Cosmos DB for NoSQL
--> Store and retrieve embeddings and execute vector similarity search for semantic retrieval
Note that there are 10 practice questions (with answers) at the end of each section to help you solidify your knowledge of the material. Also, there are 4 practice tests with 30 questions each available from the hub's main page below the exam topics section.
Overview
Modern AI applications frequently need to retrieve information based on meaning, rather than simply matching exact words.
For example, suppose a user asks:
“What options are available for taking my dog on vacation?”
A traditional keyword search might look for documents containing the words dog, vacation, or travel. A semantic search system can instead identify documents discussing pet-friendly hotels, even if those documents never use the exact words in the user’s question.
This is accomplished using vector embeddings and vector similarity search.
Azure Cosmos DB for NoSQL provides integrated vector storage, indexing, and search capabilities. Applications can store embeddings directly alongside their source documents and use the VectorDistance() system function to find documents whose vectors are closest to a query vector.
For the AI-200 exam, you should understand:
- What embeddings are
- How embeddings are generated
- How embeddings are stored in Cosmos DB
- Vector embedding policies
- Vector indexing policies
flat,quantizedFlat, anddiskANN- The
VectorDistance()function - k-nearest-neighbor (kNN) searches
- Semantic retrieval
- Filtering vector searches
- Why
TOP Nis important - How vector search fits into RAG applications
- Important vector-search limitations
1. What Is a Vector Embedding?
A vector embedding is a numerical representation of information.
An embedding model converts content such as:
- Text
- Documents
- Images
- Audio
- Other supported data
into an array of numerical values.
For example, a simplified embedding might look like:
[0.12, -0.43, 0.87, 0.21, -0.09]
Real-world embedding models generally produce vectors with many more dimensions.
The important concept is that the position of an embedding in a high-dimensional mathematical space represents characteristics of the original content.
Content with similar meanings tends to have vectors that are close together.
For example:
"How can I travel with my dog?"
might be semantically close to:
"Hotels that allow pets"
even though the two sentences don’t contain the same words.
2. Embeddings Are Generated Outside Cosmos DB
Azure Cosmos DB stores and searches embeddings, but the embedding itself is typically generated by an embedding model.
For example, an application might use an embedding API such as an Azure OpenAI embedding model.
The general workflow is:
Source content | vEmbedding model | vVector embedding | vAzure Cosmos DB
For a search request:
User query | vEmbedding model | vQuery embedding | vCosmos DB vector search | vMost semantically similar documents
The stored document embedding and query embedding need to be compatible. In practice, applications should generate both using the same embedding model or a compatible embedding space.
3. Storing Embeddings in Cosmos DB
One of the major advantages of the integrated vector capabilities in Azure Cosmos DB for NoSQL is that the embedding can be stored alongside the original document.
For example:
{ "id": "doc001", "category": "travel", "title": "Pet-Friendly Hotels", "content": "Hotels that welcome dogs and cats...", "embedding": [ 0.123, -0.456, 0.789, 0.234 ]}
The application therefore doesn’t need to maintain a completely separate database containing the vector and another database containing the associated document.
The vector and its source data can be colocated.
This is particularly useful for AI applications because the application can retrieve both:
- The similarity result
- The original content needed to answer the user’s question
from the same Cosmos DB item.
4. What Is Semantic Retrieval?
Semantic retrieval means finding information based on its meaning rather than simply matching keywords.
Consider these two documents:
Document A
“Our resort provides accommodations for guests traveling with pets.”
Document B
“Our resort has a swimming pool and fitness center.”
A user searches:
“Where can I stay with my dog?”
Document A is likely to have a much closer semantic relationship to the query.
A vector search system identifies that relationship by comparing embeddings.
The basic process is:
- Generate embeddings for documents.
- Store the embeddings with the documents.
- Generate an embedding for the user’s query.
- Compare the query vector with document vectors.
- Rank documents according to similarity.
- Return the most relevant documents.
This is the foundation of many retrieval-augmented generation (RAG) applications.
5. Vector Search in Azure Cosmos DB
Azure Cosmos DB for NoSQL provides vector search capabilities through:
- Vector embedding policies
- Vector indexing policies
- The
VectorDistance()system function
Vector indexes improve vector-search efficiency by reducing latency and RU consumption compared with an unindexed/full-scan approach.
At a conceptual level:
Azure Cosmos DB
+---------------------+
| |
Document ---> | Original content |
| |
Embedding --> | Vector embedding |
| |
| Vector index |
| |
+----------+----------+
^
|
VectorDistance()
|
Query embedding
6. Vector Embedding Policies
A vector embedding policy describes the vector properties that Cosmos DB should treat as embeddings.
The policy can specify characteristics such as:
- The vector property path
- Number of dimensions
- Distance function
- Data type
The policy establishes how Cosmos DB should interpret the vector data.
A simplified conceptual configuration might look like:
{ "vectorEmbeddings": [ { "path": "/embedding", "dataType": "float32", "dimensions": 1536, "distanceFunction": "cosine" } ]}
The exact configuration supported depends on the current Cosmos DB capabilities and account configuration, but the important exam concept is:
The vector embedding policy describes the characteristics of the vector data.
Don’t confuse this with the vector indexing policy.
7. Vector Indexing Policies
The vector indexing policy determines how Cosmos DB indexes the vectors for vector search.
Azure Cosmos DB for NoSQL currently provides three primary vector index types:
| Index | General purpose |
|---|---|
flat | Exact/brute-force vector search |
quantizedFlat | Quantized vector search for smaller/scoped workloads |
diskANN | Efficient approximate vector search for larger workloads |
Choosing the appropriate index is an important architectural decision.
8. The flat Vector Index
The flat index performs a brute-force comparison of vectors.
Its major advantage is accuracy.
A flat search can provide exact nearest-neighbor results.
However, it has a maximum vector dimensionality of 505 dimensions, which makes it unsuitable for many modern high-dimensional embedding models.
It can be appropriate for relatively small vector datasets or situations where exact recall is particularly important.
Key exam concept
Flat = exact/brute-force search.
9. The quantizedFlat Vector Index
quantizedFlat compresses vectors before storing them in the vector index.
This can provide:
- Lower latency
- Higher throughput
- Lower RU consumption
compared with an ordinary flat index.
The trade-off is that quantization can result in some loss of accuracy.
quantizedFlat supports vectors up to 4,096 dimensions.
Microsoft currently describes quantizedFlat as particularly appropriate for smaller or more narrowly scoped searches, with 50,000 vectors or fewer in the search scope being a useful general guideline—not an absolute limit. Actual workloads should be benchmarked.
Key exam concept
quantizedFlat = compressed/brute-force search with improved efficiency and a possible small accuracy trade-off.
10. The diskANN Vector Index
diskANN is designed for efficient approximate vector search, particularly for larger workloads.
It can provide:
- Low latency
- High throughput
- Efficient RU consumption
- High retrieval accuracy
It supports vectors up to 4,096 dimensions.
Microsoft describes DiskANN as generally the most performant option when the search scope exceeds approximately 50,000 vectors, although actual workload testing remains important.
Key exam concept
diskANN = approximate vector search optimized for larger datasets/search scopes.
11. Vector Index Comparison
For exam preparation, remember the following:
| Characteristic | flat | quantizedFlat | diskANN |
|---|---|---|---|
| Search type | Exact/brute force | Quantized brute force | Approximate |
| Maximum dimensions | 505 | 4,096 | 4,096 |
| Accuracy | Exact | Slight possible loss | High, configurable trade-offs |
| Large datasets | Poor fit | Better for smaller/scoped data | Excellent |
| Latency at scale | Higher | Moderate | Lower |
| RU efficiency at scale | Lower | Better | Better |
| Typical use | Small/exact searches | Smaller/scoped searches | Large-scale vector search |
12. Important Requirement: Vector Index Configuration
A vector index must be configured for the vector property that will be searched.
For example:
"vectorIndexes": [ { "path": "/embedding", "type": "diskANN" }]
The vector embedding policy and vector index work together.
A useful way to remember the distinction is:
Embedding policy = What is my vector?
Vector index = How should I search my vector?
13. Performing Vector Similarity Search
The primary Cosmos DB function used for vector similarity search is:
VectorDistance()
A basic query might look like:
SELECT TOP 10 c.title, VectorDistance(c.embedding, @queryVector) AS SimilarityScoreFROM cORDER BY VectorDistance(c.embedding, @queryVector)
This query:
- Takes the query vector.
- Compares it with
c.embedding. - Calculates a vector distance.
- Sorts the results.
- Returns the top 10 results.
Microsoft specifically recommends using TOP N for vector searches because returning unnecessary results increases RU consumption and latency.
14. Understanding VectorDistance()
The function conceptually compares:
Document vector | vVectorDistance() ^ |Query vector
The result represents the distance between the vectors.
The exact interpretation depends on the configured distance function.
Common distance concepts include:
- Cosine
- Euclidean
- Dot product
The application should use the distance function appropriate for the embedding model and workload.
15. Why Distance Matters
Suppose the query embedding is:
Q = [0.2, 0.3, 0.5]
and the database contains:
A = [0.2, 0.3, 0.5]B = [0.8, 0.1, 0.2]C = [-0.4, 0.7, 0.1]
The vector closest to the query is likely the most semantically similar.
The search engine can therefore rank results:
1. Document A2. Document B3. Document C
The application doesn’t have to know the meaning represented by every dimension.
The embedding model and vector-distance calculation handle that mathematical representation.
16. Always Use TOP N
A particularly important exam and practical-development point is:
Use
TOP Nwith vector searches.
For example:
SELECT TOP 5 c.id, c.title, VectorDistance(c.embedding, @queryVector) AS scoreFROM cORDER BY VectorDistance(c.embedding, @queryVector)
If the application only needs the five most relevant documents, there’s little reason to retrieve thousands of results.
Returning unnecessary results can increase:
- RU consumption
- Latency
- Network traffic
- Application processing
Microsoft explicitly recommends TOP N for vector searches.
17. Filtering Vector Searches
Vector search can also be combined with traditional query filtering.
For example:
SELECT TOP 10 c.title, c.category, VectorDistance(c.embedding, @queryVector) AS scoreFROM cWHERE c.category = "travel"ORDER BY VectorDistance(c.embedding, @queryVector)
This means:
Find the most semantically similar documents within the travel category.
This is extremely useful in real applications.
Examples include:
- Search products within a specific department.
- Search documents belonging to a specific tenant.
- Search hotel information within a particular region.
- Search only documents that a user is authorized to access.
Azure Cosmos DB supports combining vector search with other query filtering capabilities.
18. Vector Search and Partitioning
Azure Cosmos DB applications should always consider partitioning.
For example, a multi-tenant application might have:
{ "id": "doc123", "tenantId": "tenantA", "title": "Company policy", "embedding": [...]}
A query could restrict retrieval to a particular tenant:
SELECT TOP 10 c.title, VectorDistance(c.embedding, @queryVector) AS scoreFROM cWHERE c.tenantId = @tenantIdORDER BY VectorDistance(c.embedding, @queryVector)
This can narrow the search scope and can be important for both performance and data isolation.
19. Semantic Search vs. Keyword Search
It is important to understand the difference.
Keyword search
A keyword search primarily asks:
Does this document contain the requested word or phrase?
For example:
"automobile"
might fail to find a document that only says:
"car"
Semantic search
Semantic search asks:
Which documents are mathematically closest in meaning to this query?
Therefore:
"automobile"
may retrieve documents discussing:
carsvehiclesmotor vehiclestransportation
depending on how the embedding model represents the concepts.
20. Hybrid Search
Vector search doesn’t have to replace traditional search.
Many AI applications use hybrid search, combining:
- Keyword/full-text search
- Vector similarity
- Metadata filtering
For example:
User query | +--------------------+ | | v vKeyword search Vector search | | +---------+----------+ | v Combined ranking | v Relevant results
This can provide better retrieval than relying exclusively on either keyword or vector search.
For example, vector search is good at identifying semantic similarity, while keyword search can be valuable when an exact product ID, name, or technical term matters.
21. Vector Search and RAG
One of the most important practical applications of vector search is Retrieval-Augmented Generation (RAG).
A simplified RAG architecture looks like this:
DOCUMENT INGESTION
|
v
Generate embeddings
|
v
Azure Cosmos DB
+----------------------+
| Documents |
| Embeddings |
| Vector index |
+----------------------+
^
|
Vector retrieval
|
|
User question --> Generate embedding
|
v
Vector similarity search
|
v
Relevant documents
|
v
LLM
|
v
Generated answer
The vector database is responsible for retrieving relevant information.
The LLM is responsible for generating the final response using that retrieved information.
This distinction is important.
Vector search retrieves information; the LLM generates the response.
22. Keeping Embeddings Synchronized
Suppose the source document changes:
Original document | vEmbedding A
The document is updated:
Updated document | vEmbedding A <-- stale!
The embedding may no longer accurately represent the document.
Therefore, applications should have a mechanism to regenerate embeddings when source content changes.
Azure Cosmos DB’s change feed can be used as part of an architecture that detects changes and triggers embedding regeneration. The current AI-200 training material specifically includes change-feed processing for keeping embeddings synchronized.
A common architecture is:
Document updated | vCosmos DB change feed | vProcessing component | vGenerate new embedding | vUpdate Cosmos DB item
23. Vector Index Limitations You Should Know
Several limitations are particularly relevant for the AI-200 exam.
Maximum dimensions
Current limits include:
flat: 505 dimensionsquantizedFlat: 4,096 dimensionsdiskANN: 4,096 dimensions
Minimum vectors for quantizedFlat and diskANN
quantizedFlat and diskANN require at least 1,000 vectors for indexed vector searching. If fewer than 1,000 vectors are present, a full scan can be performed instead.
Shared throughput
Vector indexing and search currently aren’t supported on accounts using shared throughput.
Vector policy changes
Vector embedding and vector indexing policy settings aren’t simply modified in place. Depending on the specific configuration, the existing policy/index must be removed and recreated, or a new container may be required.
Vector search cannot simply be disabled
Once vector indexing and search are enabled on a container, it cannot simply be disabled.
24. Common Exam Traps
Trap 1: Confusing embeddings with indexes
An embedding is the numerical representation of content.
An index is the structure used to efficiently search those vectors.
Trap 2: Thinking Cosmos DB generates the embedding
Cosmos DB stores and searches embeddings.
An embedding model, such as an embedding API, generates the embedding.
Trap 3: Assuming diskANN is exact
diskANN is an approximate nearest-neighbor approach.
It is designed to provide excellent performance while maintaining high retrieval quality.
Trap 4: Assuming quantizedFlat is exact
Quantization can introduce a small loss of accuracy.
Trap 5: Forgetting TOP N
A vector search should generally use TOP N to avoid unnecessarily expensive retrieval.
Trap 6: Using flat for a 1,536-dimensional embedding
The current flat limit is 505 dimensions.
A 1,536-dimensional embedding requires a vector index type supporting that dimensionality, such as quantizedFlat or diskANN.
Trap 7: Treating vector search as keyword search
Vector search is based on semantic similarity, not exact text matching.
25. Exam-Focused Summary
For AI-200, remember this chain:
Source data | vEmbedding model | vVector embedding | vCosmos DB document | vVector embedding policy | vVector index | vVectorDistance() | vTOP N results | vSemantic retrieval
The most important concepts are:
| Concept | Remember |
|---|---|
| Embedding | Numerical representation of content |
| Vector store | Stores and retrieves embeddings |
| Vector embedding policy | Defines characteristics of vectors |
| Vector index | Makes vector searches more efficient |
flat | Exact/brute-force; max 505 dimensions |
quantizedFlat | Quantized; max 4,096 dimensions |
diskANN | Approximate, efficient large-scale search; max 4,096 dimensions |
VectorDistance() | Performs vector distance calculation |
TOP N | Limits results and helps control RU/latency |
| Semantic search | Finds content by meaning |
| Metadata filtering | Narrows the search space |
| Hybrid search | Combines lexical and vector retrieval |
| RAG | Uses retrieved context to augment LLM generation |
| Change feed | Can trigger embedding refresh when data changes |
Practice Exam Questions
Question 1
An AI application stores product descriptions in Azure Cosmos DB for NoSQL. The application needs to find products that are semantically similar to a user’s natural-language query.
What should the application do?
A. Store the product descriptions as strings and use CONTAINS() exclusively.
B. Generate embeddings for the product descriptions and store the vectors with the documents.
C. Convert each product description to a partition key.
D. Store each word as a separate Cosmos DB item.
Answer: B
Explanation:
Semantic retrieval requires converting content into vector embeddings. The embeddings can then be stored alongside the original documents in Cosmos DB and compared with a query embedding. Keyword functions such as CONTAINS() don’t provide semantic similarity.
Question 2
An application uses a 1,536-dimensional embedding model and needs an efficient vector index for a large production dataset.
Which vector index type is the most appropriate choice?
A. flat
B. hash
C. range
D. diskANN
Answer: D
Explanation:diskANN supports vectors up to 4,096 dimensions and is designed for efficient approximate vector search at larger scales. flat is limited to 505 dimensions and therefore cannot index a 1,536-dimensional vector.
Question 3
An application needs the five most semantically similar documents to a query vector.
Which query pattern should be used?
A.
SELECT *FROM cORDER BY VectorDistance(c.embedding, @queryVector)
B.
SELECT TOP 5 *FROM cORDER BY c.embedding
C.
SELECT TOP 5 *FROM cORDER BY VectorDistance(c.embedding, @queryVector)
D.
SELECT *FROM cWHERE c.embedding = @queryVector
Answer: C
Explanation:VectorDistance() calculates the distance between the stored embedding and query vector. TOP 5 limits the results to the five most relevant documents and helps avoid unnecessary RU consumption and latency.
Question 4
Which statement best describes the purpose of a vector embedding?
A. It is a Cosmos DB authentication token.
B. It is the partition key automatically generated by Cosmos DB.
C. It is a numerical representation of the semantic characteristics of content.
D. It is an index containing document metadata.
Answer: C
Explanation:
An embedding is a numerical representation generated by an embedding model. Semantically related content tends to produce vectors that are close together in vector space.
Question 5
A company has a relatively small vector search workload and wants to use a vector index that compresses vectors to improve efficiency while accepting a possible small loss in accuracy.
Which index should it consider?
A. flat
B. quantizedFlat
C. diskANN
D. NoSQL range indexing
Answer: B
Explanation:quantizedFlat compresses vectors before indexing. This can improve latency, throughput, and RU efficiency compared with flat, at the potential cost of some accuracy. It is particularly suited to smaller or more narrowly scoped searches.
Question 6
An application has documents containing both an embedding and a category property. It needs to find the most semantically similar documents, but only within the "finance" category.
Which approach is appropriate?
A. Perform a vector search without filtering and discard non-finance results afterward.
B. Store each category in a separate Cosmos DB account.
C. Use VectorDistance() together with a WHERE filter for the category.
D. Replace the embeddings with category names.
Answer: C
Explanation:
Vector search can be combined with traditional Cosmos DB query filters. The application can use a WHERE clause to restrict the search to documents matching the required metadata.
Question 7
A developer changes the text of a document but continues using the embedding that was generated from the old version.
What is the primary problem?
A. The partition key automatically changes.
B. The vector index is deleted.
C. The document becomes unreadable.
D. The embedding may no longer accurately represent the document.
Answer: D
Explanation:
An embedding represents the content used to generate it. If the source content changes substantially, the old embedding can become stale. Applications can use mechanisms such as the Cosmos DB change feed to detect changes and trigger embedding regeneration.
Question 8
Which statement correctly describes the flat vector index in Azure Cosmos DB for NoSQL?
A. It performs exact/brute-force vector search and supports vectors up to 505 dimensions.
B. It performs approximate DiskANN search and supports 4,096 dimensions.
C. It compresses vectors and always produces approximate results.
D. It is used only for keyword searches.
Answer: A
Explanation:
The flat index performs brute-force vector search and can provide exact nearest-neighbor results. Its current maximum vector dimensionality is 505.
Question 9
An AI application uses vector search as part of a RAG architecture.
What is the primary purpose of the vector search portion of the architecture?
A. Generate the final natural-language response.
B. Retrieve content that is semantically relevant to the user’s query.
C. Train the large language model.
D. Replace the embedding model.
Answer: B
Explanation:
Vector search retrieves relevant information based on semantic similarity. The retrieved content can then be supplied to an LLM as context for generating the final answer. Vector retrieval and LLM generation are separate responsibilities.
Question 10
A developer creates a vector search query that returns every matching document instead of limiting the result set. The application only needs the top 10 results.
What should the developer change?
A. Remove the vector index.
B. Increase the embedding dimensionality.
C. Add a TOP 10 clause to the query.
D. Replace VectorDistance() with CONTAINS().
Answer: C
Explanation:
Vector searches should generally use TOP N to limit the number of returned results. Returning more results than the application needs can increase RU consumption and latency.
Final Exam Takeaways
If you remember only a handful of things from this topic, remember these:
- Embeddings represent the semantic characteristics of content numerically.
- An embedding model generates the embedding; Cosmos DB stores and searches it.
- Embeddings can be stored alongside the original Cosmos DB document.
VectorDistance()is the key function for vector similarity searches.- Use
TOP Nwhen performing vector retrieval. flatprovides exact/brute-force search but is limited to 505 dimensions.quantizedFlatprovides a more efficient quantized approach for smaller/scoped searches.diskANNis designed for efficient approximate search at larger scales and supports up to 4,096 dimensions.- Vector search can be combined with metadata filters and hybrid search.
- Vector retrieval is a fundamental building block for RAG applications.
- When source content changes, embeddings may need to be regenerated.
- For AI-200 scenario questions, pay close attention to the dataset size, vector dimensionality, accuracy requirements, RU consumption, and latency requirements when selecting a vector index.
Go to the AI-200 Exam Prep Hub main page
The Data Community 