Windows+MSVC+scipy>=1.8.0 → segfault

Let’s consider sample program and nuitka call for compilation on Windows with MSVC: https://github.com/belonesox/bug-nuitka-scipy-180-crash

• pipenv-based setup-demo.bat for setup pipenv (pls, change your python location on set-python-path.bat).
• demo_build.bat for build
• trivial test.py

---

For linux — OK. For scipy <= 1.7.3 → OK

For scipy >=1.8.0RC1 → test.exe crashed on import scipy.stats or from scipy.stats import hmean.

---

Unhandled exception at 0x00D11CBA in test.exe: 0xC0000005: Access violation reading location 0x00000010.

[Inline Frame] test.exe!ATTACH_TRACEBACK_TO_EXCEPTION_VALUE(_object * exception_tb, _traceback *) Line 704
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\include\nuitka\exceptions.h(704)
test.exe!impl_scipy$_lib$_docscrape$$$function__14___init__(const Nuitka_FunctionObject * self, _object * * python_pars) Line 4935
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\module.scipy._lib._docscrape.cpp(4935)
test.exe!CALL_FUNCTION_WITH_ARGS3(_object * called, _object * const * args) Line 1755
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersCalling2.c(1755)
test.exe!CALL_METHOD_WITH_ARGS3(_object * source, _object * attr_name, _object * const * args) Line 10935
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersCalling2.c(10935)
test.exe!impl_scipy$_lib$_docscrape$$$function__42___init__(const Nuitka_FunctionObject * self, _object * * python_pars) Line 24421
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\module.scipy._lib._docscrape.cpp(24421)
test.exe!Nuitka_CallMethodFunctionPosArgs(const Nuitka_FunctionObject * function, _object * object, _object * const * args, int args_size) Line 2376
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\CompiledFunctionType.cpp(2376)
test.exe!CALL_FUNCTION_WITH_SINGLE_ARG(_object * called, _object * arg) Line 594
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersCalling2.c(594)
test.exe!impl_scipy$stats$_axis_nan_policy$$$function__6__axis_nan_policy_factory$$$function__3_axis_nan_policy_decorator(const Nuitka_FunctionObject * self, _object * * python_pars) Line 6567
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\module.scipy.stats._axis_nan_policy.cpp(6567)
test.exe!CALL_FUNCTION_WITH_SINGLE_ARG(_object * called, _object * arg) Line 359
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersCalling2.c(359)
test.exe!modulecode_scipy$stats$_stats_py(_object * module, const Nuitka_MetaPathBasedLoaderEntry * loader_entry) Line 160886
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\module.scipy.stats._stats_py.cpp(160886)
test.exe!loadModule(_object * module, _object * module_name, const Nuitka_MetaPathBasedLoaderEntry * entry) Line 975
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(975)
test.exe!_EXECUTE_EMBEDDED_MODULE(_object * module, _object * module_name, const char * name) Line 1014
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1014)
[Inline Frame] test.exe!EXECUTE_EMBEDDED_MODULE(_object * module) Line 1087
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1087)
test.exe!_path_unfreezer_exec_module(_object * self, _object * args, _object * kwds) Line 1423
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1423)
[External Code]
python38.dll![Frames below may be incorrect and/or missing, no symbols loaded for python38.dll]
test.exe!CALL_FUNCTION_WITH_ARGS5(_object * called, _object * const * args) Line 3240
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersCalling2.c(3240)
test.exe!IMPORT_MODULE5(_object * module_name, _object * globals, _object * locals, _object * import_items, _object * level) Line 134
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersImport.c(134)
test.exe!modulecode_scipy$stats(_object * module, const Nuitka_MetaPathBasedLoaderEntry * loader_entry) Line 852
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\module.scipy.stats.cpp(852)
test.exe!loadModule(_object * module, _object * module_name, const Nuitka_MetaPathBasedLoaderEntry * entry) Line 975
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(975)
test.exe!_EXECUTE_EMBEDDED_MODULE(_object * module, _object * module_name, const char * name) Line 1014
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1014)
[Inline Frame] test.exe!EXECUTE_EMBEDDED_MODULE(_object * module) Line 1087
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1087)
test.exe!_path_unfreezer_exec_module(_object * self, _object * args, _object * kwds) Line 1423
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1423)
[External Code]
test.exe!CALL_FUNCTION_WITH_ARGS5(_object * called, _object * const * args) Line 3240
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersCalling2.c(3240)
test.exe!IMPORT_MODULE5(_object * module_name, _object * globals, _object * locals, _object * import_items, _object * level) Line 134
	at C:\Users\stas\.virtualenvs\nuitka-scipy-stats-crash-vzkNlNj9\Lib\site-packages\nuitka\build\static_src\HelpersImport.c(134)
test.exe!modulecode___main__(_object * module, const Nuitka_MetaPathBasedLoaderEntry * loader_entry) Line 8768
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\module.__main__.cpp(8768)
test.exe!loadModule(_object * module, _object * module_name, const Nuitka_MetaPathBasedLoaderEntry * entry) Line 975
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(975)
test.exe!_EXECUTE_EMBEDDED_MODULE(_object * module, _object * module_name, const char * name) Line 1014
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1014)
[Inline Frame] test.exe!IMPORT_EMBEDDED_MODULE(const char *) Line 1068
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MetaPathBasedLoader.cpp(1068)
test.exe!EXECUTE_MAIN_MODULE(const char * module_name) Line 416
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MainProgram.cpp(416)
test.exe!wmain(int argc, wchar_t * * argv) Line 1101
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\static_src\MainProgram.cpp(1101)
[External Code]
test.exe!modulecode_scipy$stats$_continuous_distns(_object * module, const Nuitka_MetaPathBasedLoaderEntry * loader_entry) Line 448047
	at C:\projects\bugs\nuitka\nuitka-scipy-stats-crash\test.build\module.scipy.stats._continuous_distns.cpp(448047)

Line 4935 of module.scipy._lib._docscrape.cpp

    ATTACH_TRACEBACK_TO_EXCEPTION_VALUE(exception_keeper_value_1, exception_keeper_tb_1);

corresponds to 148 line in scipy\_lib\_docscrape.py

        try:
            self._parse()     # ←←←← here 
        except ParseError as e:
            e.docstring = orig_docstring
            raise

All generated sources attached → test-build.zip

(another version … test-build.zip )

---

Crash not depens:

• nuitka version (reproduced on all since 0.7.0, including factory, but factory got another fail → build.log, not tested before 0.7.0)
• Python version (reproduced on w32 python 3.8.10, w64 python 3.10.0)

---

Not tested — mingw GCC.

I can provide access for VM with Windows, if it helps…

[kayhayen]

It is incredibly slow to compile with MinGW64, what is relatively acceptable with MSVC, takes hours to complete with it. I am doing it, because I have hopes for better tracebacks. But I guess, my plan on this one is to make sure all CPython test suites run with 0.8 and debug python versions, in vague hopes of catching it under defined conditions with reference counting tests.

[kayhayen]

I thought I reproduced on Linux, but then on investigation it never happened again. These corruption bugs are very susceptible to what the memory released ends up becoming used for next time, and this depends on allocation patterns. Windows memory allocator is more often exposing issues of Nuitka, while not making it easy to debug.

Regarding pipenv install, I am getting complaints that it doesn’t work for branches to update them, pipenv will complain after I push over commits, I guess in the lock file, the commit is is stored, or hash of archive mismatches, something like that. The installing from branch with pipenv ought to work, but I am not sure if it is working or not actually. I have no idea what {editable = true, ref = "factory", git = "https://github.com/Nuitka/Nuitka.git"} means to pipenv. Does it address the issue with it changing that I was talking about?

Supporting pipenv installation is important to me. I love the tool. Make a new report if there is anything wrong. I have to admit, I don’t have test coverage of it.