smart_open() printing lots of output during transfer when not desired
See original GitHub issueWhen I use smart_open() (connecting to s3) it prints a ton of stuff like this:
...
133
142
170
154
144
137
147
...
Can I turn that off? What is it? There’s a lot of it, I don’t want it to print this.
Also some docs around the project would be greatly appreciated. For example another question (because there’s no docs):
How do I know if the s3 connection is using multipart upload or uploading serially? Maybe you guys always have it do multipart? I wouldn’t know though, all I see is it seems like you have to get some parameters correct for it to use multipart upload
Issue Analytics
- State:
- Created 5 years ago
- Comments:9 (6 by maintainers)
Top Results From Across the Web
Python3 is not printing the output in the desired order when ...
Just use subprocess.check_output to get the output that the command would print. import subprocess, shutil print ("Current running kernel ...
Read more >Untitled
Python S3 StreamSimple AWS S3 Logging in Python3 Using Boto3. this files from the storage solution was application/octet-stream.
Read more >Transforming data for your application with S3 Object Lambda
Walk through an example of how to transform data in Amazon S3 using S3 Object Lambda to suit the needs of the requesting...
Read more >smart-open - PyPI
Why? Working with large remote files, for example using Amazon's boto3 Python library, is a pain. boto3's Object.upload_fileobj() ...
Read more >Your Guide to the Python print() Function
As you just saw, calling print() without arguments results in a blank line, ... which doesn't contain any characters at all, not even...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Turns out this is expected behavior. The Python interpreter outputs the result of the last command, if it is not
None
. For example:The .write method returns the number of bytes written. So
You can suppress that by assigning to a throwaway variable:
I have smart-open 1.8.0. I installed it with
pip3 install smart-open
. I didn’t make any edits to the source code.I was thinking it would be nice to use smart-open to stream from a url into s3 w/o writing the data to a local area first. That’s why I used it to open two sources and read from one and write the reads into the other. It’s quite handy if it can work.
I couldn’t get it to print in a similar way when I did them separately. And there’s nothing special about 44413, that’s just how many lines of ouput I got when doing the example code I provided above. I gave the count instead of pasting all of the lines, especially since they’re all the same. i.e. they are all just a number around 150.