[BUG] ClientRetryPolicy broken when executing stored procedure

Describe the bug When executing a stored procedure ( e.g. bulk-delete.js) via AsyncDocumentClient and the request runs into rate limiting then the retry mechanism throws an error.

Here is the log output: (from SDK 3.3.1)

11:19:35.814 [reactor-http-epoll-4] ERROR c.a.d.c.internal.ClientRetryPolicy - locationEndpoint is null because ClientRetryPolicy::onBeforeRequest(.) is not invoked, probably request creation failed due to invalid options, serialization setting, etc.
11:19:35.817 [reactor-http-epoll-4] ERROR c.a.d.c.i.RenameCollectionAwareClientRetryPolicy - onBeforeSendRequest is not invoked, encountered failure due to request being null
com.azure.data.cosmos.CosmosClientException: Message: {"Errors":["Request rate is large"]}
...

To Reproduce Execute bulk-delete.js stored procedure on a large enough dataset so that the execution runs into execution bounds (rate limiting). In my case there were 500 documents to delete and the partition was configured with 400 RU/s. In the first execution 345 documents were successfully deleted. The described error occurred after executing the stored procedure a second time.

Expected behavior The retry is performed automatically and the remaining documents are getting deleted. Basically the same behaviour as for standard queries using the SDK.

In version com.microsoft.azure:azure-cosmosdb:2.4.4 everything was working correctly. That was the log output:

2019-05-14 13:03:33.363  WARN [-,,,] 28633 --- [o-eventloop-4-3] c.m.a.c.r.i.ResourceThrottleRetryPolicy  : Operation will be retried after 8569 milliseconds. Current attempt 1, Cumulative delay PT8.569S Exception: {}

Actual behavior Error is thrown (see log output above) and no retry is performed.

Environment summary First discovered in SDK Version: com.microsoft.azure:azure-cosmosdb:2.4.5. Same for the 3.0.0-beta and also 3.3.1. Currently we are using 3.5.0, still the same issue.

The original issue was created here: https://github.com/Azure/azure-cosmosdb-java/issues/118

@kushagraThapar asked me to create a new issue here. I also updated the description to reflect the current issue status.

[xinlian12]

fix has merged into master. The fix will be included in 4.2.1.

[kushagraThapar]

@yito88 - we are taking a look at this - thanks for the suggestion.

@xinlian12 is taking a look at this and will get a fix out soon.