Random Access Violations on shutdown

Environment

• Pythonnet version: 3.0.0.post1
• Python version: 3.8.10
• Operating System: Windows 11
• .NET Runtime: .NET Framework 472

Details

• Describe what you were trying to get done.

  I am getting random access violations during shutdown in pythonnet. The issue can be re-produced with a C# function as simple as

public static int Add(int a, int b) => a + b;

I get the access violation; however, it is not deterministic at all. Some times, it succeeds as expected, other times I get this. Note that I am upgrading from pythonnet 2.5.2 to pythonnet 3.0.0.post1 so this code used to work. All I am doing is invoking this code from python. I am running this python code that uses pythonnet in pytest. I haven’t seen this when we were using pythonnet 2.5.2.

• If there was a crash, please include the traceback here.

Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Python.Runtime.Runtime.PyObject_TYPE(BorrowedReference op)
   at Python.Runtime.Runtime.NullGCHandles(IEnumerable`1 objects)
   at Python.Runtime.Runtime.TryCollectingGarbage(Int32 runs, Boolean forceBreakLoops)
   at Python.Runtime.Runtime.Shutdown()
   at Python.Runtime.PythonEngine.Shutdown()
   at Python.Runtime.Loader.Shutdown(IntPtr data, Int32 size)

[siegfriedpammer]

Yes, please add an option to disable the shutdown code completely. I have had cases where simply Ctrl+Cing the process caused a .NET stack trace to be printed, which should not be the case. I have tried monkey-patching the pythonnet.unload function, but it made everything worse, that is, even more annoying stack traces printed to the console.

[siegfriedpammer]

Would this be an appropriate fix?

diff --git a/src/runtime/Finalizer.cs b/src/runtime/Finalizer.cs
index 713564f..d6bbc3b 100644
--- a/src/runtime/Finalizer.cs
+++ b/src/runtime/Finalizer.cs
@@ -229,6 +229,7 @@ namespace Python.Runtime

                         IntPtr copyForException = obj.PyObj;
                         Runtime.XDecref(StolenReference.Take(ref obj.PyObj));
+                        CLRObject.reflectedObjects.Remove(copyForException);
                         collected++;
                         try
                         {
@@ -236,7 +237,7 @@ namespace Python.Runtime
                         }
                         catch (Exception e)
                         {
-                            HandleFinalizationException(obj.PyObj, e);
+                            HandleFinalizationException(copyForException, e);
                         }
                     }

The fix is simple: remove a possible reference to the currently disposed object from the reflectedObjects set. If the item does not exist the Remove is a no-op.

I am not entirely sure why a variable copyForException is not used in the catch block.