stderr duplication failed in console scripts

Description

I use a console_scripts entry point with a test function. If I do pip install . or use setup.py to make a binary dist then install it, under some shell like Git Bash, running the created test-script.exe. stdin, stdout and stderr should all be set to a TextIOWrapper however, stderr is set to None.

The stderr handle is apparently not correctly set.

This is probably related to https://github.com/pypa/pip/issues/10444 and the fix introduced in Distlib 0.3.4 for a similar issue.

Expected behavior

$ test-script.exe
stdin=<_io.TextIOWrapper name='<stdin>' mode='r' encoding='cp1252'>
stdout=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp1252'>
stderr=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='cp1252'>

pip version

22.0

Python version

Python 3.7

OS

Windows

How to Reproduce

# setup.py
from setuptools import setup

setup(
    name="test",
    packages=["testing"],
    entry_points={
        "console_scripts": [
            "test-script=testing:main"
        ]
    }
)

# testing/__init__.py
import sys


def main():
    print(f"stdin={sys.stdin}")
    print(f"stdout={sys.stdout}")
    print(f"stderr={sys.stderr}")

Output

# With pip 21.*
$ test-script.exe
stdin=<_io.TextIOWrapper name='<stdin>' mode='r' encoding='cp1252'>
stdout=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp1252'>
stderr=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='cp1252'>

# With pip 22.0, 22.0.1 and 22.0.2 
$ test-script.exe
stdin=<_io.TextIOWrapper name='<stdin>' mode='r' encoding='cp1252'>
stdout=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp1252'>
stderr=None

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pradyunsg]

So the old version of distlib broke certain GUI use cases by making error dialog boxes pop up.

The new version of distlib broke other – CLI this time – use cases by making them error out silently.

This is good context, thanks for noting this here @eli-schwartz!

I think having a visible (albeit cryptic) error that users can discover is better than a silent error that they can’t. Further, CLI entry points are a lot more common than GUI entry points and I’m pretty sure the right thing to do here is to downgrade distlib to 0.3.3 until @vsajip is able able to fix this issue in distlib.

I’ll cut a bugfix for this.

[pierreluctg]

@carlkl I understand what the problem is, and I understand what we’d need to do to fix it (revert to an older distlib, or wait for a distlib fix). What I don’t know is how many people are affected, nor do I know whether this issue is more disruptive than the issue that the distlib upgrade fixed, or whether it will be more disruptive than the cost of making a bugfix release.

The comment you linked to says it is a “severe” bug. It’s not severe for me, I’ve never seen it happen. So if you can be more precise about what circumstances cause it to occur, that would make it easier to assess the severity in a way that doesn’t just boil down to “it doesn’t happen for me” vs “it happens for a program someone else uses a lot”. You also say “Now sys.stderr is None if you start launchers created by pip 22”. My reproducer above demonstrates that this isn’t always true. So the question is how often is that the case, and under what circumstances. I can’t tell, because as yet I haven’t ever had it happen for me.

🤷 I’m not really that bothered. Ultimately someone needs to make the case that this is important enough for us to produce a bugfix release with a reverted distlib. I’m only trying to help people make that case - if they don’t, then this will likely just wait for a fix in distlib, and the next pip release.

I have an example of a very common use case affected by this bug.

Imagine a python developer using Jenkins to run his unit tests, using pytest, on a Windows host.

Now because he start using pip 22.x (he updated because pip warned him that he was not using the latest version), running pytest will not produce any output on the console, tests will not run and there is no reporting.

Now how is this user supposed to root cause this issue? There is no output at all since stderr is None.

And this is not a intermittent issue, in the affected environment (like Jenkins on Windows in this example), stderr will always be None.

@pfmoore, I hope this helps you understand the importance of getting this bug fixed.