line buffering (buffering=1) warning on Python 3.8

Since the recent change made in #1014, I got a warning like below when launching images based on Python 3.8 (python=3.8 in environment.yml).

/srv/conda/envs/notebook/lib/python3.8/subprocess.py:848: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used

I believe this issue is related to the change in Python 3.8 where it started complaining whereas previous versions silently ignored buffering=1 for binary mode. I guess the related code is below:

https://github.com/jupyterhub/repo2docker/blob/a5f5bbbb75a9945d1f8fe8f8ff4844dfd4481742/repo2docker/buildpacks/repo2docker-entrypoint#L40-L46

Not sure if related, I also noticed texts on the console (mostly) lost colors after this recent change; Jupyter log messages used to have colored headings for warning/info/etc, but they are now all monochrome. Yet some texts are still printed in colors (i.e. Julia banner lost colors, but its prompt still has a color).

[manics]

Before we fix the TTY issue we should decide what beahaviour we want.

If you’re running repo2docker from the command line with default options it’ll output the logs to the console, in which case colour logs makes sense, and doing this through a pty(?) makes sense since in principle someone could provide input via stdin. If it’s run as a background container or a Kubernetes pod you wouldn’t expect a terminal to be connected so non-colour logs makes sense.

[tomyun]

1. Somehow I failed to include a complete warning message.

/srv/conda/envs/notebook/lib/python3.8/subprocess.py:848: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdout = io.open(c2pread, 'rb', bufsize)

The warning about buffering=1 is coming from io.open() accepts buffering which is set to bufsize when io.open() is called in subprocess.Popen(). AFAICT, it’s not about a typo between buffering and bufsize, but more about the value itself.

1. Regarding color output, I guess it’s about the difference between pipe and tty. For example, Julia running on a regular terminal clearly shows it’s on TTY.

julia> Sys.stdout
Base.TTY(RawFD(14) open, 0 bytes waiting)

On the other hand, Julia running on an image created by the latest repo2docker has stdout mapped to pipe which is as expected with the recent change.

julia> Sys.stdout
Base.PipeEndpoint(RawFD(12) open, 0 bytes waiting)

It turns out Julia initializes colored stdout for non-TTY only when it’s given an explicit option (color=yes, not default auto) that explains my experience.

It’s likely Jupyter (and IPython) works in a similar way that they ended up getting a non-color output here. Seems like this kind of issue is not uncommon as Python has a package like ptyprocess.