Uploading files using the Java SDK results in a Resetting to invalid mark error when part size is more than 2gb.

We are building the Put objects argument builder as follows:

PutObjectArgs.builder() .bucket("test-bucket") .object("test-identifier") .stream(stream, -1, 3000000000) .contentType("application/octet-stream") .build();

This works well for both Minio server and Amazon S3 but we noticed that when we set the part size to more than 2gb, in this case its 3000000000 it throws an error when uploading.

Part sizes lesser than or equal to 2gb works.

The stream we are passing is a bufferedinputstream.

Uploading via presigned urls works even if the part size is more than 2gb.

This happens whether the file size we are uploading is greater than or lesser than the part size.

Expected Behavior

Should be able to upload a file in Minio/S3

Current Behavior

Upload fails with the error:

Caused by: java.io.IOException: Resetting to invalid mark at java.base/java.io.BufferedInputStream.reset(BufferedInputStream.java:454) at io.minio.S3Base.getAvailableSize(S3Base.java:1230) at io.minio.S3Base.putObject(S3Base.java:1134) at io.minio.MinioClient.putObject(MinioClient.java:1666)

Possible Solution

Steps to Reproduce (for bugs)

Context

Regression

Your Environment

Minio 8.2.1

[klauspost]

@balamurugana I am not java expert, but it would seem to me that you are overflowing the int conversion when > 2G here:

https://github.com/minio/minio-java/blob/836359242f62c5add495a873d26a7c071f182a02/api/src/main/java/io/minio/S3Base.java#L1218

So sending negative size?

[balamurugana]

@balamurugana Is the buffer essential to have the same size as the part size?

@klauspost FYI https://github.com/minio/minio-java/issues/1204