Better reporting on output from PEP 517 build backends

Currently, pip install and pip wheel on a PEP 517 compliant source tree hides all output on stdout and stderr from the build backend. Only if an error occurs, both are printed (though it sometimes misses parts of stdout). This means you can’t show the user any information about the progress of long compilations (or even that it’s still progressing and hasn’t got stuck anywhere). It also hides any warnings.

Is there any designated way to communicate from build backends to the user, and if not, would it be possible for pip to forward the stdout and/or stderr from build backends?

Running with -v is also problematic because it contains lots of noise, duplicates the build backend’s output in case of errors and prints an irrelevant stack trace. (Apparently, there’s no difference between -v, -vv and -vvv with pip install). The current output capturing also means that all tools checking for a tty won’t output any color, making error and log messages less readable.

[pradyunsg]

Running with -v is also problematic because it contains lots of noise, duplicates the build backend’s output in case of errors and prints an irrelevant stack trace.

These issues are no longer a concern. We’ve improved handling of output from the build-backends in 22.0 and also reduced the verbosity of a single -v flag in releases since this issue was filed.

I’m going to go ahead and close this now – I think we’re in a good spot w.r.t. output handling from build backends, but if folks think there’s specific improvements we can make, I’d like you to chip in at https://discuss.python.org/t/13779/ (you can log in via GitHub) since the relevant changes here likely affect more tools than just pip (at the very least, it affects build backends).

[cjerdonek]

It seems like six different issues or proposed improvements are being mentioned:

1. Verbose mode logs too many messages
2. The setup.py clean traceback isn’t needed
3. The InstallationError traceback at the end isn’t needed
4. The subprocess build output (stdout and/or stderr) shouldn’t require verbose mode to display
5. The build output should forward directly to the console instead of going through pip when displayed (e.g. to take advantage of tty)
6. ~The beginning and end of the build output should be distinguished / delineated better~ (This was addressed by PR #6655.)

To keep the discussions manageable, I think it would be best to separate these into different issues. Each one can be discussed separately, and any resolution / PR if it gets to that point could likely be implemented independent of the PR for any other. Here it seems like the interest is mainly in (4) and (5). I would suggest creating a new issue for each one you think should be addressed.

I started by creating an issue for (2) here: https://github.com/pypa/pip/issues/6642 (But I’m not completely sure it’s an issue after starting to investigate.)