Cannot read input zip file because of Sentry IncludeRequestPayload is activated

We have an API controller with the following action:

public async Task<IActionResult> UnpackZipFile()
{
    try
    {
        using (var zip = new ZipArchive(this.Request.Body))
        {
            // Process ZIP file
        }
        return new OkResult();
    }
    catch (Exception ex)
    {
        this._logger.LogError(ex, "Could not unpack zip file");
        return new BadRequestResult();
    }
}

When Sentry’s IncludeRequestPayload is false, the action behaves correctly. this.Request.Body is of type Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream in this case.

When Sentry’s IncludeRequestPayload is true, the using part throws a System.NotSupportedException exception, with the message “The content has not been fully buffered yet”. This time this.Request.Body is of type Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream. Moreover, if the file is too large the log is not even sent to Sentry!

I suspect the Sentry middleware to wrap the request body stream in a rewindable FileBufferingReadStream, but this stream is incompatible with the ZipArchive constructor. I have no clue whether I’m doing something wrong in my original code, or if this is a bug the Sentry team should fix.

I am using ASP.NET Core 2.2 with IIS in-process hosting model and Sentry.AspNetCore NuGet in version 1.1.2.

[bruno-garcia]

@madmox Thanks for the repro! I was able to simulate the error when enabling the buffering of requests.

As it turns out, the issue is not related to the Sentry SDK directly. Yes it is the SDK that enables request buffering with EnableRewind but that might as well be done by another framework feature or package.

The SDK doesn’t do anything with the request before it hits your controller. It only calls EnableRewind to be able to Seek(Begin) only in case there’s an exception.

The issue is actually with FileBufferingReadStream Seek which will throw if you try to Seek the end of the stream when the underlying Stream has not being read yet in combination with ZipArchive which starts reading the stream from the end.

I assume they chose to throw here because the alternative would be to block the thread while buffering.

One way to work around this is simply consuming the Body stream before passing it to ZipArchive.

Would be great if FileBufferingReadStream had some API like: await Request.Body.BufferAsync(); where you can await asynchronously for it to read the whole request but there’s no such API. Again, maybe because there’s no nice way to make one.

Some work around could look like (which is inefficiently duplicating all data over to a MemoryStream):

using (var mem = new MemoryStream())
{
    await this.Request.Body.CopyToAsync(mem);
    using (var zip = new ZipArchive(mem)) // Won't throw anymore
    {
        entriesCount = zip.Entries.Count;
    }
}

Or just reading it into some small throw-away buffer. The data is anyway buffered in the FileBufferingReadStream. Still wasted copying and allocations (Task) going on though:

var buffer = new byte[1024]; // magic buffer size
var re = 0;
do
{
    re = await this.Request.Body.ReadAsync(buffer, 0, buffer.Length);
} while (re > 0);
this.Request.Body.Position = 0;
// Free to use this.Request.Body now with ZipArchive

Lets call those work arounds for now as it seems the ideal solution here is offering an API in the Sentry SDK to avoid enabling request body rewinding altogether. But instead of whitelisting a user-defined Content-Type list, we can go with a single callback that gets invoked in the SentryMiddleware before calling EnableRewind giving your the whole Request, or something else. Needs some discussion I guess.

Important to note that this issue will end up hitting more people, unrelated to Sentry’s SDK.

[bruno-garcia]

This was resolved with the option MaxRequestBodySize:

https://github.com/getsentry/sentry-dotnet/blob/1ab7c273d569b356c2f32a0d16fac9749275507e/src/Sentry.AspNetCore/SentryMiddleware.cs#L89-L92