CosmosDB emulator: set default number of partitions to 1 #5641

radical · 2024-09-10T02:58:42Z

This helps improve the container startup time for the emulator.

Note that log shows n+1 partitions being started. For example with AZURE_COSMOS_EMULATOR_PARTITION_COUNT=1 the container logs show:

Started 1/2 partitions
Started 2/2 partitions
Started

Fixes #2177 .

davidfowl · 2024-09-10T03:04:53Z

#2177

mitchdenny · 2024-09-10T03:05:33Z

@radical it might be that 2 is the minimum so may as well set it there to avoid confusion?

mitchdenny · 2024-09-10T03:07:38Z

Would be good to add a test to ensure that if someone does this - they can override the value we provides:

    c.WithEnvironment("AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "30");
});

radical · 2024-09-10T03:12:18Z

@radical it might be that 2 is the minimum so may as well set it there to avoid confusion?

When I set it to 5, 6 are started - Started 1/6 partitions.

radical · 2024-09-10T04:24:44Z

Interesting failure:

Reason: ({"code":"ServiceUnavailable","message":"Sorry, we are currently experiencing high demand in this region South Central US, and cannot fulfill your request at this time Tue, 10 Sep 2024 04:08:43 GMT.\r\nActivityId: d7bcb021-32ae-4875-a07a-6d9a80908cff, Microsoft.Azure.Documents.Common/2.14.0"}

davidfowl · 2024-09-10T04:29:00Z

That message came from the emulator?

radical · 2024-09-10T04:49:51Z

That message came from the emulator?

A transient exception occurred during execution. The operation will be retried after 0ms.
Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: ServiceUnavailable (503); Substatus: 1007; ActivityId: 1a191f5e-2c77-4731-88bc-20e35c4c16dc; Reason: ({"code":"ServiceUnavailable","message":"Sorry, we are currently experiencing high demand in this region South Central US, and cannot fulfill your request at this time Tue, 10 Sep 2024 04:04:57 GMT.\r\nActivityId: 1a191f5e-2c77-4731-88bc-20e35c4c16dc, Microsoft.Azure.Documents.Common/2.14.0"}
RequestUri: https://127.0.0.1:44089/dbs/ef/colls;
RequestMethod: POST;
Header: Authorization Length: 80;
Header: x-ms-date Length: 29;
Header: x-ms-cosmos-sdk-supportedcapabilities Length: 1;
Header: x-ms-activity-id Length: 36;
Header: Cache-Control Length: 8;
Header: User-Agent Length: 112;
Header: x-ms-version Length: 10;
Header: Accept Length: 16;
Header: traceparent Length: 55;
, Request URI: /dbs/ef/colls, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Linux/22.04 cosmos-netstandard-sdk/3.34.4);
   at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)
   at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
--- Cosmos Diagnostics ---{"Summary":{"GatewayCalls":{"(503, 1007)":1}},"name":"CreateContainerStreamAsync","start datetime":"2024-09-10T04:04:57.261Z","duration in milliseconds":204.6975,"data":{"Client Configuration":{"Client Created Time Utc":"2024-09-10T04:04:57.0562050Z","MachineId":"vmId:69b1b058-5394-499f-a350-3837a3edeeef","VM Region":"westus","NumberOfClientsCreated":2,"NumberOfActiveClients":2,"ConnectionMode":"Gateway","User Agent":"cosmos-netstandard-sdk/3.42.0|2|X64|Ubuntu 22.04.4 LTS|.NET 8.0.7|DN| Microsoft.EntityFrameworkCore.Cosmos/8.0.8","ConnectionConfig":{"gw":"(cps:50, urto:120, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:True, be:False)"},"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":2}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":203.9043,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":203.8816,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-10T04:04:53.6506560Z","cpu":0.000,"memory":1860140.000,"threadInfo":{"isThreadStarving":"no info","availableThreads":32765,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":203.869,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":203.8547,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":203.5386,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":203.5346,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":190.3148,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-10T04:04:57.2625039Z","DurationInMs":141.8954,"RequestUri":"https://127.0.0.1:44089/dbs/ef/colls","ResourceType":"Collection","HttpMethod":"POST","ActivityId":"1a191f5e-2c77-4731-88bc-20e35c4c16dc","StatusCode":"ServiceUnavailable","ReasonPhrase":"Service Unavailable"}]},"Point Operation Statistics":{"Id":"PointOperationStatistics","ActivityId":"1a191f5e-2c77-4731-88bc-20e35c4c16dc","ResponseTimeUtc":"2024-09-10T04:04:57.4640883Z","StatusCode":503,"SubStatusCode":1007,"RequestCharge":0,"RequestUri":"dbs/ef","ErrorMessage":"Microsoft.Azure.Documents.DocumentClientException: {\"code\":\"ServiceUnavailable\",\"message\":\"Sorry, we are currently experiencing high demand in this region South Central US, and cannot fulfill your request at this time Tue, 10 Sep 2024 04:04:57 GMT.\\r\\nActivityId: 1a191f5e-2c77-4731-88bc-20e35c4c16dc, Microsoft.Azure.Documents.Common/2.14.0\"}\nRequestUri: https://127.0.0.1:44089/dbs/ef/colls;\nRequestMethod: POST;\nHeader: Authorization Length: 80;\nHeader: x-ms-date Length: 29;\nHeader: x-ms-cosmos-sdk-supportedcapabilities Length: 1;\nHeader: x-ms-activity-id Length: 36;\nHeader: Cache-Control Length: 8;\nHeader: User-Agent Length: 112;\nHeader: x-ms-version Length: 10;\nHeader: Accept Length: 16;\nHeader: traceparent Length: 55;\n, Request URI: /dbs/ef/colls, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Linux/22.04 cosmos-netstandard-sdk/3.34.4\n   at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)\n   at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)","RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]}]}
Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: ServiceUnavailable (503); Substatus: 1007; ActivityId: 1a191f5e-2c77-4731-88bc-20e35c4c16dc; Reason: ({"code":"ServiceUnavailable","message":"Sorry, we are currently experiencing high demand in this region South Central US, and cannot fulfill your request at this time Tue, 10 Sep 2024 04:04:57 GMT.\r\nActivityId: 1a191f5e-2c77-4731-88bc-20e35c4c16dc, Microsoft.Azure.Documents.Common/2.14.0"}
RequestUri: https://127.0.0.1:44089/dbs/ef/colls;
RequestMethod: POST;
Header: Authorization Length: 80;
Header: x-ms-date Length: 29;
Header: x-ms-cosmos-sdk-supportedcapabilities Length: 1;
Header: x-ms-activity-id Length: 36;
Header: Cache-Control Length: 8;
Header: User-Agent Length: 112;
Header: x-ms-version Length: 10;
Header: Accept Length: 16;
Header: traceparent Length: 55;
, Request URI: /dbs/ef/colls, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Linux/22.04 cosmos-netstandard-sdk/3.34.4);
   at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)
   at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
--- Cosmos Diagnostics ---{"Summary":{"GatewayCalls":{"(503, 1007)":1}},"name":"CreateContainerStreamAsync","start datetime":"2024-09-10T04:04:57.261Z","duration in milliseconds":204.6975,"data":{"Client Configuration":{"Client Created Time Utc":"2024-09-10T04:04:57.0562050Z","MachineId":"vmId:69b1b058-5394-499f-a350-3837a3edeeef","VM Region":"westus","NumberOfClientsCreated":2,"NumberOfActiveClients":2,"ConnectionMode":"Gateway","User Agent":"cosmos-netstandard-sdk/3.42.0|2|X64|Ubuntu 22.04.4 LTS|.NET 8.0.7|DN| Microsoft.EntityFrameworkCore.Cosmos/8.0.8","ConnectionConfig":{"gw":"(cps:50, urto:120, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:True, be:False)"},"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":2}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":203.9043,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":203.8816,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-10T04:04:53.6506560Z","cpu":0.000,"memory":1860140.000,"threadInfo":{"isThreadStarving":"no info","availableThreads":32765,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":203.869,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":203.8547,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":203.5386,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":203.5346,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":190.3148,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-10T04:04:57.2625039Z","DurationInMs":141.8954,"RequestUri":"https://127.0.0.1:44089/dbs/ef/colls","ResourceType":"Collection","HttpMethod":"POST","ActivityId":"1a191f5e-2c77-4731-88bc-20e35c4c16dc","StatusCode":"ServiceUnavailable","ReasonPhrase":"Service Unavailable"}]},"Point Operation Statistics":{"Id":"PointOperationStatistics","ActivityId":"1a191f5e-2c77-4731-88bc-20e35c4c16dc","ResponseTimeUtc":"2024-09-10T04:04:57.4640883Z","StatusCode":503,"SubStatusCode":1007,"RequestCharge":0,"RequestUri":"dbs/ef","ErrorMessage":"Microsoft.Azure.Documents.DocumentClientException: {\"code\":\"ServiceUnavailable\",\"message\":\"Sorry, we are currently experiencing high demand in this region South Central US, and cannot fulfill your request at this time Tue, 10 Sep 2024 04:04:57 GMT.\\r\\nActivityId: 1a191f5e-2c77-4731-88bc-20e35c4c16dc, Microsoft.Azure.Documents.Common/2.14.0\"}\nRequestUri: https://127.0.0.1:44089/dbs/ef/colls;\nRequestMethod: POST;\nHeader: Authorization Length: 80;\nHeader: x-ms-date Length: 29;\nHeader: x-ms-cosmos-sdk-supportedcapabilities Length: 1;\nHeader: x-ms-activity-id Length: 36;\nHeader: Cache-Control Length: 8;\nHeader: User-Agent Length: 112;\nHeader: x-ms-version Length: 10;\nHeader: Accept Length: 16;\nHeader: traceparent Length: 55;\n, Request URI: /dbs/ef/colls, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Linux/22.04 cosmos-netstandard-sdk/3.34.4\n   at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)\n   at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)\n   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)","RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]}]}

Request URI: /dbs/ef/colls -

aspire/playground/CosmosEndToEnd/CosmosEndToEnd.ApiService/Program.cs

Lines 51 to 59 in 6df3fef

    
           app.MapGet("/ef", async (TestCosmosContext context) => 
        
           { 
        
               await context.Database.EnsureCreatedAsync(); 
        
               context.Entries.Add(new EntityFrameworkEntry()); 
        
               await context.SaveChangesAsync(); 
        
               return await context.Entries.ToListAsync(); 
        
           });

Error json expanded:

{
   "Summary" : {
      "GatewayCalls" : {
         "(503, 1007)" : 1
      }
   },
   "children" : [
      {
         "children" : [
            {
               "children" : [
                  {
                     "children" : [
                        {
                           "children" : [
                              {
                                 "children" : [
                                    {
                                       "children" : [
                                          {
                                             "data" : {
                                                "Client Side Request Stats" : {
                                                   "AddressResolutionStatistics" : [],
                                                   "ContactedReplicas" : [],
                                                   "FailedReplicas" : [],
                                                   "HttpResponseStats" : [
                                                      {
                                                         "ActivityId" : "1a191f5e-2c77-4731-88bc-20e35c4c16dc",
                                                         "DurationInMs" : 141.8954,
                                                         "HttpMethod" : "POST",
                                                         "ReasonPhrase" : "Service Unavailable",
                                                         "RequestUri" : "https://127.0.0.1:44089/dbs/ef/colls",
                                                         "ResourceType" : "Collection",
                                                         "StartTimeUTC" : "2024-09-10T04:04:57.2625039Z",
                                                         "StatusCode" : "ServiceUnavailable"
                                                      }
                                                   ],
                                                   "Id" : "AggregatedClientSideRequestStatistics",
                                                   "RegionsContacted" : [],
                                                   "StoreResponseStatistics" : []
                                                },
                                                "Point Operation Statistics" : {
                                                   "ActivityId" : "1a191f5e-2c77-4731-88bc-20e35c4c16dc",
                                                   "BELatencyInMs" : null,
                                                   "ErrorMessage" : "Microsoft.Azure.Documents.DocumentClientException: 

                                                   {"code":"ServiceUnavailable","message":"Sorry, we are currently experiencing high demand in this region South Central US, and cannot fulfill your request at this time Tue, 10 Sep 2024 04:04:57 GMT.\\r\
ActivityId: 1a191f5e-2c77-4731-88bc-20e35c4c16dc, Microsoft.Azure.Documents.Common/2.14.0"}
RequestUri: https://127.0.0.1:44089/dbs/ef/colls;
RequestMethod: POST;
Header: Authorization Length: 80;
Header: x-ms-date Length: 29;
Header: x-ms-cosmos-sdk-supportedcapabilities Length: 1;
Header: x-ms-activity-id Length: 36;
Header: Cache-Control Length: 8;
Header: User-Agent Length: 112;
Header: x-ms-version Length: 10;
Header: Accept Length: 16;
Header: traceparent Length: 55;
, Request URI: /dbs/ef/colls, RequestStats: Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum, SDK: Linux/22.04 cosmos-netstandard-sdk/3.34.4
   at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)
   at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)",

                                                   "Id" : "PointOperationStatistics",
                                                   "RequestCharge" : 0,
                                                   "RequestSessionToken" : null,
                                                   "RequestUri" : "dbs/ef",
                                                   "ResponseSessionToken" : null,
                                                   "ResponseTimeUtc" : "2024-09-10T04:04:57.4640883Z",
                                                   "StatusCode" : 503,
                                                   "SubStatusCode" : 1007
                                                }
                                             },
                                             "duration in milliseconds" : 190.3148,
                                             "name" : "Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request"
                                          }
                                       ],
                                       "duration in milliseconds" : 203.5346,
                                       "name" : "Microsoft.Azure.Cosmos.Handlers.TransportHandler"
                                    }
                                 ],
                                 "duration in milliseconds" : 203.5386,
                                 "name" : "Microsoft.Azure.Cosmos.Handlers.RouterHandler"
                              }
                           ],
                           "duration in milliseconds" : 203.8547,
                           "name" : "Microsoft.Azure.Cosmos.Handlers.RetryHandler"
                        }
                     ],
                     "duration in milliseconds" : 203.869,
                     "name" : "Microsoft.Azure.Cosmos.Handlers.TelemetryHandler"
                  }
               ],
               "data" : {
                  "System Info" : {
                     "systemHistory" : [
                        {
                           "cpu" : 0,
                           "dateUtc" : "2024-09-10T04:04:53.6506560Z",
                           "memory" : 1860140,
                           "numberOfOpenTcpConnection" : 0,
                           "threadInfo" : {
                              "availableThreads" : 32765,
                              "isThreadStarving" : "no info",
                              "maxThreads" : 32767,
                              "minThreads" : 2
                           }
                        }
                     ]
                  }
               },
               "duration in milliseconds" : 203.8816,
               "name" : "Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler"
            }
         ],
         "duration in milliseconds" : 203.9043,
         "name" : "Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler"
      }
   ],
   "data" : {
      "Client Configuration" : {
         "Client Created Time Utc" : "2024-09-10T04:04:57.0562050Z",
         "ConnectionConfig" : {
            "gw" : "(cps:50, urto:120, p:False, httpf: False)",
            "other" : "(ed:True, be:False)",
            "rntbd" : "(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)"
         },
         "ConnectionMode" : "Gateway",
         "ConsistencyConfig" : "(consistency: NotSet, prgns:[], apprgn: )",
         "MachineId" : "vmId:69b1b058-5394-499f-a350-3837a3edeeef",
         "NumberOfActiveClients" : 2,
         "NumberOfClientsCreated" : 2,
         "ProcessorCount" : 2,
         "User Agent" : "cosmos-netstandard-sdk/3.42.0|2|X64|Ubuntu 22.04.4 LTS|.NET 8.0.7|DN| Microsoft.EntityFrameworkCore.Cosmos/8.0.8",
         "VM Region" : "westus"
      }
   },
   "duration in milliseconds" : 204.6975,
   "name" : "CreateContainerStreamAsync",
   "start datetime" : "2024-09-10T04:04:57.261Z"
}

This is happening during CreateContainerStreamAsync (which is this?). Does this mean that efcore/cosmos is trying to create a real Azure Cosmos container? Do we need something on our side to have it use the emulator?

radical · 2024-09-10T04:54:14Z

The failure is unrelated to this PR, IIUC, so shouldn't block the merge.

radical · 2024-09-10T06:22:17Z

/azp run

azure-pipelines · 2024-09-10T06:22:29Z

Azure Pipelines successfully started running 1 pipeline(s).

eerhardt · 2024-09-10T15:15:52Z

It would be good to hear from @Pilchie @kirankumarkolli on this change.

kirankumarkolli · 2024-09-10T17:55:46Z

My understanding is emulator AZURE_COSMOS_EMULATOR_PARTITION_COUNT impacts the concurrent #RUs supported.
Test suits needs to be proofed be with-in those bounds.

Checking with SME.

radical · 2024-09-18T06:57:01Z

My understanding is emulator AZURE_COSMOS_EMULATOR_PARTITION_COUNT impacts the concurrent #RUs supported. Test suits needs to be proofed be with-in those bounds.

Checking with SME.

@kirankumarkolli Any update on this?

davidfowl · 2024-09-24T15:38:31Z

You can rebuild on top of #5157 @radical

kirankumarkolli · 2024-09-24T18:47:18Z

Apologies for delay.

Recommendation is to use with /disableRateLimiting without it collections created with > 10K result in throttling (for devops it might not be an issue but good practice to set-up).

radical · 2024-09-24T21:02:26Z

You can rebuild on top of #5157 @radical

I'll open a new PR to use the API from #5157 to change the partition count for the tests.

radical · 2024-09-24T21:03:21Z

Apologies for delay.

Recommendation is to use with /disableRateLimiting without it collections created with > 10K result in throttling (for devops it might not be an issue but good practice to set-up).

@davidfowl do we want to use /disableRateLimiting in the product?

radical · 2024-09-26T06:41:26Z

Recommendation is to use with /disableRateLimiting without it collections created with > 10K result in throttling (for devops it might not be an issue but good practice to set-up).

@kirankumarkolli What does "collections created with > 10k" mean? 10k of what?

Pilchie · 2024-09-26T15:15:10Z

10k of what

Cosmos DB Request Units (RUs). 10,000 is the maximum for a single partition.

davidfowl · 2024-09-26T16:28:37Z

@davidfowl do we want to use /disableRateLimiting in the product?

Yes. Lets do it.

CosmosDB emulator: set default number of partitions to 1

7403b32

dotnet-issue-labeler bot added the area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication label Sep 10, 2024

radical requested review from mitchdenny, davidfowl, sebastienros and eerhardt September 10, 2024 02:59

Address review feedback from @ mitchdenny - add tests

6a8c1bf

davidfowl approved these changes Sep 10, 2024

View reviewed changes

davidfowl added area-integrations Issues pertaining to Aspire Integrations packages azure Issues associated specifically with scenarios tied to using Azure and removed area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication labels Sep 10, 2024

fix typo in test

6df3fef

This was referenced Sep 10, 2024

[tests] Improve Cosmos EndToEnd test #5639

Closed

Add WithPartitionCount and WithDataVolume to Azure.Hosting.Azure.CosmosDB #5157

Merged

build-analysis bot mentioned this pull request Sep 11, 2024

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Open

3 tasks

build-analysis bot mentioned this pull request Sep 22, 2024

Flaky Playground test for Cosmos /ef endpoint #5415

Open

radical added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Sep 24, 2024

radical mentioned this pull request Sep 26, 2024

Add /disableRateLimiting to cosmos integration #5962

Open

radical closed this Sep 26, 2024

radical deleted the cosmos-emulator-default-partitions branch September 26, 2024 21:42

github-actions bot locked and limited conversation to collaborators Oct 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CosmosDB emulator: set default number of partitions to 1 #5641

CosmosDB emulator: set default number of partitions to 1 #5641

radical commented Sep 10, 2024 •

edited

Loading

davidfowl commented Sep 10, 2024

mitchdenny commented Sep 10, 2024

mitchdenny commented Sep 10, 2024

radical commented Sep 10, 2024

radical commented Sep 10, 2024

davidfowl commented Sep 10, 2024

radical commented Sep 10, 2024

radical commented Sep 10, 2024

radical commented Sep 10, 2024

azure-pipelines bot commented Sep 10, 2024

eerhardt commented Sep 10, 2024

kirankumarkolli commented Sep 10, 2024

radical commented Sep 18, 2024 •

edited

Loading

davidfowl commented Sep 24, 2024

kirankumarkolli commented Sep 24, 2024 •

edited

Loading

radical commented Sep 24, 2024

radical commented Sep 24, 2024

radical commented Sep 26, 2024

Pilchie commented Sep 26, 2024

davidfowl commented Sep 26, 2024

CosmosDB emulator: set default number of partitions to 1 #5641

CosmosDB emulator: set default number of partitions to 1 #5641

Conversation

radical commented Sep 10, 2024 • edited Loading

davidfowl commented Sep 10, 2024

mitchdenny commented Sep 10, 2024

mitchdenny commented Sep 10, 2024

radical commented Sep 10, 2024

radical commented Sep 10, 2024

davidfowl commented Sep 10, 2024

radical commented Sep 10, 2024

radical commented Sep 10, 2024

radical commented Sep 10, 2024

azure-pipelines bot commented Sep 10, 2024

eerhardt commented Sep 10, 2024

kirankumarkolli commented Sep 10, 2024

radical commented Sep 18, 2024 • edited Loading

davidfowl commented Sep 24, 2024

kirankumarkolli commented Sep 24, 2024 • edited Loading

radical commented Sep 24, 2024

radical commented Sep 24, 2024

radical commented Sep 26, 2024

Pilchie commented Sep 26, 2024

davidfowl commented Sep 26, 2024

radical commented Sep 10, 2024 •

edited

Loading

radical commented Sep 18, 2024 •

edited

Loading

kirankumarkolli commented Sep 24, 2024 •

edited

Loading