Download large files

When using async semantics to get a large file, the whole file must be read into RAM, also blocking a thread. This is especially an issue in a server environment.

One might expect the following to work:

    IO::Stream stream;
	
    await m_client.GetObjectAsync(this.m_bucket,
                path,
                s => stream = s,
                cancellationToken);
	
    await data = stream.read() ...

However that fails because the stream “s” must be consumed inside the stream callback. This is because RestSharp closes the request immediately after calling the callback. There is actually a closed issue over there on that:

https://github.com/restsharp/RestSharp/issues/539

While that may be an “edge case” for RestSharp, it is clearly an issue here where we could easily be dealing with large files. The only way to read a file is as in the example:

    IO::MemoryStream stream;
	
    await m_client.GetObjectAsync(this.m_bucket,
                path,
                s => s.CopyTo(stream),
                cancellationToken);
	
    data = stream.read() ...

However, that uses much RAM and blocks a thread in the threadpool when being run in a service.

[KyleGobel]

Is this still planned? I thought it was strange when I first saw to access the stream you had to provide a callback, i guess that’s just because of RestSharp?

IMO this makes the client library pretty limiting.

[ptr1120]

Please consider to reopen this issue, because for the minio main use case it seems to have a high priority to support downloading files async and be able to stream files without loading the whole content into memory. A closed issue does not support improving this in near future.