Sphinx-Needs affecting incremental behavior

I am running Sphinx with the (default) incremental behavior on. This means HTMLs shall be generated only for the changed rST files. If there are no changes, a sphinx-build shall return the following.

looking for now-outdated files... none found
no targets are out of date.

I recently noticed that all HTMLs were always generated, even if no changes are done. This means, the sphinx-build returns for example,

building [html]: targets for x source files that are out of date

This makes the successive builds always slow as they are not incremental anymore.

I tried to remove the used third-party extensions one by one to nail down to the following.

If sphinx-needs is added as an extension and the needs_extra_options or needs_functions is initialized in conf.py, the incremental (write) behavior is turned off. Adding only the extension but no needs config as above in conf.py does not cause any problem. Also no rST file with a need object is required to cause the wrong behavior, just adding the above configs will rebuild all rST files always. NOTE: The read process is still incremental, but the write not anymore.

To reproduce the problem.

1. Create a conf.py with

extensions = ['sphinxcontrib.needs']

from docutils.parsers.rst import directives   
needs_extra_options = {
"my-option": directives.unchanged
}

1. Create a single rST file with any contents (no need object required)

2. Do a sphinx-build

3. Do a sphinx-build again

4. The HTML are always regenerated with newer timestamps.

The same problem happens with sphinx-test-reports. In this case, just adding the extension to conf.py is enough to cause the problem.

extensions = ['sphinxcontrib.needs', 'sphinxcontrib.test_reports']

Tested with sphinx-needs 0.7.1 and sphinx 3.5.4/sphinx 4.1.1. Even 0.6.x versions of sphinx-needs have this problem I suppose as the incremental behavior was broken since long (or always broken).

[danwos]

This warning is coming from Sphinx, telling you that the default type of needs_extra_options is a dict, because dict was supported first by Sphinx-Needs.

The change allows to set needs_extra_options now to list or dict. But for sure there can be only one default and its type. I will change it to a list, as I think this should be the common configuration type from now on.

Will also add an additional hint from Sphinx-Needs that with 0.7.2 “list” is the way to go.

[twodrops]

Cool, I can confirm that the incremental build works again with sphinx-needs! If the above reported warning is removed as well, then my build will be green too 😃 (as my sphinx reports all warnings as errors).

I will do some detailed testing soon.