question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

smart_open() printing lots of output during transfer when not desired

See original GitHub issue

When I use smart_open() (connecting to s3) it prints a ton of stuff like this:

...
133
142
170
154
144
137
147
...

Can I turn that off? What is it? There’s a lot of it, I don’t want it to print this.

Also some docs around the project would be greatly appreciated. For example another question (because there’s no docs):

How do I know if the s3 connection is using multipart upload or uploading serially? Maybe you guys always have it do multipart? I wouldn’t know though, all I see is it seems like you have to get some parameters correct for it to use multipart upload

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:9 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
mpenkovcommented, Mar 9, 2019

Turns out this is expected behavior. The Python interpreter outputs the result of the last command, if it is not None. For example:

>>> 1 + 2
3

The .write method returns the number of bytes written. So

>>> fout.write(b'hello world')
9

You can suppress that by assigning to a throwaway variable:

>>> _ = fout.write(b'hello world')
1reaction
ampersand-fivecommented, Mar 9, 2019

I have smart-open 1.8.0. I installed it with pip3 install smart-open. I didn’t make any edits to the source code.

I was thinking it would be nice to use smart-open to stream from a url into s3 w/o writing the data to a local area first. That’s why I used it to open two sources and read from one and write the reads into the other. It’s quite handy if it can work.

I couldn’t get it to print in a similar way when I did them separately. And there’s nothing special about 44413, that’s just how many lines of ouput I got when doing the example code I provided above. I gave the count instead of pasting all of the lines, especially since they’re all the same. i.e. they are all just a number around 150.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Python3 is not printing the output in the desired order when ...
Just use subprocess.check_output to get the output that the command would print. import subprocess, shutil print ("Current running kernel ...
Read more >
Untitled
Python S3 StreamSimple AWS S3 Logging in Python3 Using Boto3. this files from the storage solution was application/octet-stream.
Read more >
Transforming data for your application with S3 Object Lambda
Walk through an example of how to transform data in Amazon S3 using S3 Object Lambda to suit the needs of the requesting...
Read more >
smart-open - PyPI
Why? Working with large remote files, for example using Amazon's boto3 Python library, is a pain. boto3's Object.upload_fileobj() ...
Read more >
Your Guide to the Python print() Function
As you just saw, calling print() without arguments results in a blank line, ... which doesn't contain any characters at all, not even...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found