Skip to content

Conversation

@sourabh1007
Copy link
Contributor

@sourabh1007 sourabh1007 commented Nov 8, 2024

Description

This PR introduces the following changes:

Network-Level Telemetry Implementation

  • Telemetry Updates:
    • Initialized network meter in DocumentClient.cs for client metrics.
    • Refactored ClientContextCore.cs to ensure consistent metric and trace collection.
    • Added models for NetworkMetricData and OperationMetricData.
  • Metrics Enhancements:
    • Populated telemetry dimensions with consistent naming in key files (e.g., IActivityAttributePopulator.cs).
    • Core metric generation logic in CosmosDbNetworkMeter.cs.
    • Refactored CosmosDbOperationMeter.cs and aligned packages (TracesStabilityFactory.cs).
    • Included region info in ClientSideRequestStatisticsTraceDatum.cs.
  • Test Coverage:
    • Added and updated tests in OpenTelemetryMetricsTest.cs.

Reference: open-telemetry/semantic-conventions#1495

Type of change

  • [] New feature (non-breaking change which adds functionality)

sourabh1007 and others added 30 commits October 18, 2024 11:04
more changes

wip

rewrite meters

refactor code

dimension names

test fix

contract update

import fixes

wip

added other metrics

code refactor

add docuemnattion

Delete Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/Microsoft - Backup.Azure.Cosmos.EmulatorTests.csproj

upodated contract
…ripts (#4813)

[INTERNAL] CI: Fixes emulator set-up to leverage central SDK teams
scripts

Observations
- Lessmsi: extraction taking longer (1M) then msiexec based model
- Starting emulator from ProgramFolder path is faster 
- Unnecessary exponential retry logic (current one)
# Pull Request Template

## Description

This PR adds optional attributes in the `VectorIndexDefinition` class to
support partitioned DiskANN. A typical index definition would be
something like the below:

```
{
    "indexingPolicy": {
        "automatic": true,
        "indexingMode": "Consistent",
        "includedPaths": [
            {
                "path": "/*",
                "indexes": []
            }
        ],
        "excludedPaths": [],
        "compositeIndexes": [],
        "spatialIndexes": [],
        "vectorIndexes": [
            {
                "path": "/vector1",
                "type": "flat"
            },
            {
                "path": "/vector2",
                "type": "quantizedFlat",
                "quantizationByteSize": 3,
                "vectorIndexShardKey": [
                    "/Country"
                ]
            },
            {
                "path": "/vector3",
                "type": "diskANN",
                "quantizationByteSize": 2,
                "indexingSearchListSize": 100,
                "vectorIndexShardKey": [
                    "/ZipCode"
                ]
            }
        ]
    },
    "vectorEmbeddingPolicy": {
        "vectorEmbeddings": [
            {
                "path": "/vector1",
                "dataType": "int8",
                "dimensions": 1200,
                "distanceFunction": "dotproduct"
            },
            {
                "path": "/vector2",
                "dataType": "uint8",
                "dimensions": 3,
                "distanceFunction": "cosine"
            },
            {
                "path": "/vector3",
                "dataType": "float32",
                "dimensions": 400,
                "distanceFunction": "euclidean"
            }
        ]
    },
    "id": "test_binary_vector_container_6",
    "partitionKey": {
        "paths": [
            "/pk"
        ],
        "kind": "Hash"
    }
}
```

## Type of change

Please delete options that are not relevant.

- [x] New feature (non-breaking change which adds functionality)

## Closing issues

To automatically close an issue: closes #4628

---------

Co-authored-by: Kiran Kumar Kolli <[email protected]>
Azure.Core:  Fixed upgrading azure core dependency to 1.44.1

Changes
- ResourceType: Conflict between Azure.Core and
Microsoft.Azure.Cosmos.Documents
- Microsoft.Bcl.AsyncInterfaces: Azure.Core needs at-least 6.0.0 (as
Azure core upgraded part of minor version, we are good as well)
- Microsoft.Azure.Cosmos.Encryption.Custom: Direct dependency on
Azure.Core removed (now its transitive)
- Microsoft.Azure.Cosmos.Encryption.Custom.Performance.Tests: `NU1903`
added Newtonsoft.Json dependency
- Performance project: `NU1903` Newtonsoft dependency upgraded
…onKeyStreamAsync API to GA (#4814)

DeleteAllItemsByPartitionKeyStreamAsync: Adds
DeleteAllItemsByPartitionKeyStreamAsync API to GA

Changes
- DeleteAllItemsByPartitionKeyStreamAsync: Marked virtual with default
NotSupportedException (Dependent libraries will continue to work)
- Encryption package related changes will be in a follow-up PR after the
package is published
@sourabh1007 sourabh1007 force-pushed the users/sourabhjain/otelnetworkmetrics branch from c382c69 to f05a443 Compare December 2, 2024 18:01
Copy link
Member

@FabianMeiswinkel FabianMeiswinkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sourabh1007 sourabh1007 added the auto-merge Enables automation to merge PRs label Dec 13, 2024
Copy link
Member

@FabianMeiswinkel FabianMeiswinkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge Enables automation to merge PRs Telemetry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants