CosmosClient with wrong key throws DocumentClientException instead of CosmosException

This ticket relates with another one that was been previously closed (https://github.com/Azure/azure-cosmos-dotnet-v3/pull/785). I have basically implemented exactly what was described on that ticket. I am trying to create a cosmos client with a wrong key. I am able to get a DB and a container but as soon as I try to do a container action I get a DocumentClientException.

new CosmosClient(_appSettings.CosmosDbEndpoint, wrongKey,
       new CosmosClientOptions()
       {
           ConnectionMode = ConnectionMode.Direct,
           SerializerOptions = new CosmosSerializationOptions
           {
              PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase
            }
         });

_database = client.GetDatabase(_databaseId);
_container = _database.GetContainer(_transactionsContainerId);

try
{
   var response = await _container.CreateItemAsync(item);
   response.Resource.ETag = response.ETag;

   return response.Resource;
}
catch (CosmosException e)
{
 ...
}
catch (DocumentClientException e1)
{
...
}
catch (Exception e)
{
... The exception comes here as nothing seems to be able to catch a DocumentClientException
 }

We are using Microsoft.Azure.Cosmos 3.13.0

One of my colleagues did some research and he found that the issue might be on this line of code. It might need to be inside the try catch: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Resource/ClientContextCore.cs#L332

Some more details I have found. If we initialize the client with a valid key and then the key gets rotated, we get the CosmosException as expected with the following details: Response status code does not indicate success: Unauthorized (401); Substatus: 0; ActivityId: XXX; Reason: (Message: {"Errors":["The MAC signature found in the HTTP request is not the same as the computed signature. Server used following string to sign - XXX...

However, as I mentioned above, the DocumentClientException is still thrown if the client is initialized already with an invalid key.

[ealsur]

@j82w This reproes on Write operations:

[TestMethod]
public async Task EnsureUnauthorized_ThrowsCosmosClientException()
{
    string authKey = ConfigurationManager.AppSettings["MasterKey"];
    string endpoint = ConfigurationManager.AppSettings["GatewayEndpoint"];
    // Take the key and change some middle character
    authKey = authKey.Replace("m", "M");
    CosmosClient cosmosClient = new CosmosClient(
        endpoint,
        authKey);
    CosmosException exception = await Assert.ThrowsExceptionAsync<CosmosException>(() => cosmosClient.GetContainer("test", "test").CreateItemAsync<dynamic>(new { id = "test" }));
    Assert.AreEqual(HttpStatusCode.Unauthorized, exception.StatusCode);
}

 ActivityId: 5bea91c4-7655-461e-a21c-226c79280ff4, Microsoft.Azure.Documents.Common/2.11.0, Windows/10.0.19042 cosmos-netstandard-sdk/3.13.3
    Stack Trace:    at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\GatewayStoreClient.cs:line 117
       at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\GatewayAccountReader.cs:line 50
       at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, Func`2 getDatabaseAccountFn) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Routing\GlobalEndpointManager.cs:line 109
       at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync() in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\GatewayAccountReader.cs:line 59
       at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync() in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Resource\Settings\CosmosAccountServiceConfiguration.cs:line 60
       at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync() in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\DocumentClient.cs:line 6509
       at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\DocumentClient.cs:line 925
       at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync() in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\DocumentClient.cs:line 1406
       at Microsoft.Azure.Cosmos.DocumentClient.GetCollectionCacheAsync() in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\DocumentClient.cs:line 542
       at Microsoft.Azure.Cosmos.ContainerCore.GetCachedContainerPropertiesAsync(CancellationToken cancellationToken) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Resource\Container\ContainerCore.cs:line 310
       at Microsoft.Azure.Cosmos.ContainerCore.GetPartitionKeyPathTokensAsync(CancellationToken cancellationToken) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Resource\Container\ContainerCore.cs:line 343
       at Microsoft.Azure.Cosmos.ContainerCore.GetPartitionKeyValueFromStreamAsync(Stream stream, CancellationToken cancellation) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Resource\Container\ContainerCore.Items.cs:line 765
       at Microsoft.Azure.Cosmos.ContainerCore.ExtractPartitionKeyAndProcessItemStreamAsync[T](Nullable`1 partitionKey, String itemId, T item, OperationType operationType, ItemRequestOptions requestOptions, CosmosDiagnosticsContext diagnosticsContext, CancellationToken cancellationToken) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Resource\Container\ContainerCore.Items.cs:line 673
       at Microsoft.Azure.Cosmos.ContainerCore.CreateItemAsync[T](CosmosDiagnosticsContext diagnosticsContext, T item, Nullable`1 partitionKey, ItemRequestOptions requestOptions, CancellationToken cancellationToken) in C:\GIT\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Resource\Container\ContainerCore.Items.cs:line 72

[ealsur]

@NRam0s was right about the location for the fix, sharing PR in a min