Request Latency is missing in v3 responses

Is your feature request related to a problem? Please describe. In v2 of the library, responses had a RequestLatency field which showed the latency of the raw request against Cosmos DB. In v3, it seems that this field is no longer available. I see that internally, the library is storing this value within the specific CosmosDiagnostics class for each response type, but it’s not publicly accessible without deserializing through CosmosDiagnostics.ToString(), which is not a performant or efficient solution.

Describe the solution you’d like The Response<'T> type should expose the RequestLatency value, or else provide an easily consumable form of CosmosDiagnostics that is not internal.

Describe alternatives you’ve considered Deserializing the CosmosDiagnostics.ToString(), but the cost to do this for every single request is too heavy.

[ealsur]

@ylibrach You can get the required statistics for Autoscale from Azure Monitor: https://docs.microsoft.com/en-us/azure/cosmos-db/logging#interpret which can be read programmatically to generate an autoscale logic.

I have trouble understanding point # 1. RequestLatency is the time it takes for the request to go from the origin and get back, if there is a network blip or your connections/sockets are starved, that latency will be affected. How is that related to RU consumption and self throttling? That seems to be related to RequestCharge.

[ylibrach]

Based on the description it sounds like the application is doing the scaling. The better design here is to log the information and have a separate application relying on telemetry to do the scaling. This way the container can be scaled with multiple application instances.

We actually have both:

1. The individual microservices are throttling themselves (max parallelization) based on their own single-thread RU/s consumed over the past several minutes (this is where Request Latency comes into play), compared to total RU consumed by all microservices against a specific collection. This prevents us from sending too many requests against Cosmos and getting tons of 429’s before the collection has a chance to be scaled up to the necessary throughput.
2. There is a separate dedicated autoscaler microservice which monitors the total aggregate RUs hitting each collection across all microservices, and makes throughput adjustments for the collections accordingly.