Debug build CI job crashes on `stats._unuran` threading test

Observed in gh-14962 (unrelated PR). Details from the CI log:

sx...................................................................... [ 99%]
......................[gw0] node down: Not properly terminated
F
replacing crashed worker gw0
........................................................ss...ssssssssss. [ 99%]
.s...........................sssssssssssssssssssssssssssssssssssssssssss [ 99%]
ssssssssssssssssys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='UTF-8'>
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='UTF-8'>

=================================== FAILURES ===================================
_ build/testenv/lib/python3.8/site-packages/scipy/stats/tests/test_sampling.py _
[gw0] linux -- Python 3.8.0 /usr/bin/python3.8-dbg
worker 'gw0' crashed while running 'build/testenv/lib/python3.8/site-packages/scipy/stats/tests/test_sampling.py::test_threading_behaviour'

Likely a bug (threading behavior is something that could indeed cause this segfault), but it’s not happening on other PRs. Let’s just see if it happens again.

This test came in in commit 402ac3535b8c (main PR that added UNU.RAN, Cc @tirthasheshpatel).

[tirthasheshpatel]

Hi @rgommers,

I noticed that I was still sometimes getting attribute lookup errors. I think this is happening because Cython looks up the class attributes to find the release method of the _lock variable when I call _lock.release(). At least, that’s what the generated C code says:

Although the _lock variable is not destroyed, python doesn’t allow ANY attribute lookups when a live exception is set. The workaround is to fetch the error before calling _lock.release() and restore the error once the lock has been released:

I tried this and reran my experiment 100,000 times. It doesn’t give me an attribute lookup error anymore and I am confident this would resolve the issue.

Sorry for not noticing this before #15331 was merged! I will reopen this issue and submit another PR. At some point, it will be better to rewrite the base class in C.

[rgommers]

This happened once more (log), again on the Python38-dbg job. Output:

..sssssssssssssssssssssssssssssssssssssssss [ 99%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='UTF-8'>
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=0 mode='r' encoding='UTF-8'>

=================================== FAILURES ===================================
_ build/testenv/lib/python3.8/site-packages/scipy/stats/tests/test_sampling.py _
[gw0] linux -- Python 3.8.0 /usr/bin/python3.8-dbg
worker 'gw0' crashed while running 'build/testenv/lib/python3.8/site-packages/scipy/stats/tests/test_sampling.py::test_threading_behaviour'
...
=========================== short test summary info ============================
FAILED scipy/stats/tests/test_sampling.py::test_threading_behaviour
...
= 1 failed, 35655 passed, 2599 skipped, 114 xfailed, 7 xpassed in 662.35s (0:11:02) =

The crash seems to occur very infrequently.