Provide better experience when the BodyReader is left in a reading state

When middleware calls HttpContext.Request.BodyReader.ReadAsync() and exits without calling HttpContext.Request.BodyReader.AdvanceTo(), Kestrel will fail to read the following request while logging the following exception as a warning (at least for Content-Length HTTP/1.1 requests, not sure about others):

System.InvalidOperationException
  HResult=0x80131509
  Message=Reading is already in progress.
  Source=Microsoft.AspNetCore.Server.Kestrel.Core
  StackTrace:
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1ContentLengthMessageBody.TryReadInternal(ReadResult& readResult) in /_/src/Servers/Kestrel/Core/src/Internal/Http/Http1ContentLengthMessageBody.cs:line 126

This is pretty common type of error when something throws in between calls to ReadAsync() and AdvanceTo() and AdvanceTo() isn’t in a finally block.

It would be better if Kestrel logged a warning that explicitly tells the developer that there’s a missing call to BodyReader.AdvanceTo(). Better yet would be to recover from this situation by draining the rest of the request despite the BodyReader being left in a bad state so that the connection can still be used for future requests.

We should also ensure Kestrel handles the BodyWriter being left in a bad state gracefully.

See #17723 and #14727 (In that case it was Kestrel itself throwing without properly advancing the connection pipe though).

• ASP.NET Core version 3.1

[halter73]

I just noticed something:

Example exception we see server-side:

   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.OnConsumeAsync()

Example exception we see client-side (as mentioned, we do not know if these are related or not):

  at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)`

Given the HTTP/1.x vs HTTP/2 here, I’m going to say these are unrelated.

[davidni]

@halter73 we are seeing the same exception referenced in this issue but I audited the code and found no direct uses of Request.BodyReader, so I thought I’d raise here in case that isn’t the only way this can happen. Our repro is in IslandGateway, and (I think) the only component dealing directly with the request body is YARP. Any suggestions what might be going on here? We are investigating this as part of a weeks-long investigation into a reliability issue that impacts some cross-geo requests issued from a .NET 6.0.8 client to IslandGateway. We own both sides. In case these things are indeed related, the pattern we see is:

• Client observes: IOException: The response ended prematurely while waiting for the next frame from the server
• Server observes: InvalidOperationException: Reading is already in progress

Detailed exceptions follow. And to be clear, we have not yet established that the client errors and servers errors described here are correlated. This is just one hypothesis we are working through.

---

Example exception we see server-side:

Connection id "0HMKH7444S95V", Request id "0HMKH7444S95V:00003038": An unhandled exception was thrown by the application.

Microsoft.AspNetCore.Connections.ConnectionAbortedException: The connection was aborted by the application.
 ---> System.InvalidOperationException: Reading is already in progress.
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1ContentLengthMessageBody.TryReadInternal(ReadResult& readResult)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.OnConsumeAsync()
   --- End of inner exception stack trace ---

---

Example exception we see client-side (as mentioned, we do not know if these are related or not):

 ---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.IO.IOException: The request was aborted.
 ---> System.IO.IOException: The response ended prematurely while waiting for the next frame from the server.
   at System.Net.Http.Http2Connection.<ReadFrameAsync>g__ThrowMissingFrame|57_1()
   at System.Net.Http.Http2Connection.ReadFrameAsync(Boolean initialFrame)
   at System.Net.Http.Http2Connection.ProcessIncomingFramesAsync()
   --- End of inner exception stack trace ---
   at System.Net.Http.Http2Connection.ThrowRequestAborted(Exception innerException)
   at System.Net.Http.Http2Connection.Http2Stream.CheckResponseBodyState()
   at System.Net.Http.Http2Connection.Http2Stream.TryEnsureHeaders()
   at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)
   at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
(...)
   at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken)
   at Yarp.ReverseProxy.Forwarder.HttpForwarder.SendAsync(HttpContext context, String destinationPrefix, HttpMessageInvoker httpClient, ForwarderRequestConfig requestConfig, HttpTransformer transformer)
   --- End of inner exception stack trace ---
(...)