needs_warnings not returning error code or writing to error log

Usage of -W shall handle all warnings as errors and the sphinx-build executable shall return a non-zero value as error code. Also the usage of -w <file> shall write all warnings to a file.

This works incase of Sphinx warnings, but not for warnings created with needs_warnings. Because of this pipelines cannot be configured to break incase of needs_warnings.

To reproduce the problem, do the following steps

1. Create needs_warnings in conf.py
2. Create a need warning within any rST
3. Execute sphinx-build as follows from a console

sphinx-build -M html docs _build -W --keep-going -w error.log  

1. Observe need_warnings on the console
2. Execute the following from the console

echo $?

1. 0 is returned (a non-zero value is expected)
2. Check the file error.log
3. Only the following line is present

WARNING: Sphinx-Needs warnings were raised. See console / log output for details.

All needs_warnings are expected in the log.

Could it be that needs_warnings is always writing to stdout instead of stderror (even with -W)?

[danwos]

@haiyangToAI figured out the reason: The status code calculation happens in Sphinx far too early and is based on a warning_count. See https://github.com/sphinx-doc/sphinx/blob/81a4fd973d4cfcb25d01a7b0be62cdb28f82406d/sphinx/application.py#L345 As our warning-checks happen in the cleanup-phase, which gets executed by Sphinx after the status code calculation, the value for status_code may not be valid anymore.

@haiyangToAI is currently checking if we can make a dirty-hack and change this value on our own.

[twodrops]

@haiyangToAI @danwos How can we conclude this topic? My current understanding from the above discussion is that, sphinx-needs warnings (or in general warnings thrown by sphinx extensions?) are treated differently compared to sphinx warnings. Is this an issue coming sphinx or sphinx-needs?