question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

You are not using the length param in Minio.put_object, why not make it optional?

See original GitHub issue

See: https://github.com/minio/minio-py/blob/master/minio/api.py#L797

    def put_object(self, bucket_name, object_name, data, length,
                   content_type='application/octet-stream',
                   metadata=None):
        # ...
        return self._do_put_object(bucket_name, object_name,
                                   current_data, len(current_data),
                                   metadata=metadata)

And getting the length of when doing put_object is tedious and make no much sense.

For example:

buf = io.BytesIO(b'Hello, World!')
length = buf.getbuffer().nbytes
ossc.put_object('default', 'test-blob', buf, length)

Which makes too much noise to my code, I probably should care whether the buf object is an BytesIO or other file-like object.

Thanks!

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
balamuruganacommented, Nov 19, 2020

@gerritsangel The support is added in the master https://github.com/minio/minio-py/blob/master/minio/api.py#L1252

We will make new release soon.

0reactions
gerritsangelcommented, Nov 19, 2020

Hello,

I was wondering on how to upload a stream of unknown size with the Minio Python SDK.

The Java SDK supports uploading of unknown size streams, so I’m a bit surprised that the Python SDK doesn’t seem to support it.

(See https://minio-java.min.io/io/minio/MinioClient.html#putObject-io.minio.PutObjectArgs-):

 // Upload unknown sized input stream.
 minioClient.putObject(
     PutObjectArgs.builder().bucket("my-bucketname").object("my-objectname").stream(
             inputStream, -1, 10485760)
         .contentType("video/mp4")
         .build());

Here, the file part size must be manually specified, but the stream length can be set as -1.

is there any protocol issue that would prevent the Python SDK to support unknown streams? Reading the entire stream into memory is difficult if it is a large file, especially on devices with few memory 😦

Read more comments on GitHub >

github_iconTop Results From Across the Web

Python Client API Reference — MinIO Object Storage for Linux
(Optional) Credentials provider of your account in S3 service. NOTE on concurrent usage: Minio object is thread safe when using the Python threading...
Read more >
PutObject - Amazon Simple Storage Service
With server-side encryption, Amazon S3 encrypts your data as it writes it to disks in its data centers and decrypts the data when...
Read more >
Is it possible to save a group of files with MinIO client in one ...
You can try use this S3 feature, MinIO also support this feature. Create .tar or .zip archive and send to S3 with metadata...
Read more >
AWS S3 with Java - Reactive Support - Baeldung
By using a reactive stack, we can make our service much less ... we need the part's length, but chunked file transfers do...
Read more >
S3Fs — S3Fs 2022.11.0+4.g5917684 documentation
s3.read_block(path, offset=1000, length=10, delimiter=b'\n') b'A whole line ... If you are not using async-style programming, you do not need to know about ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found