Py.GIL hangs on second function call.

Environment

• Pythonnet version: v3.0.0-preview2021-10-01
• Python version: 3.8
• Operating System: Windows 10
• .NET Runtime: net5.0
• IDE: Jetbrains Rider RD-211.7142.19

Details

• Describe what you were trying to get done.

  The code below runs fine on the first function call, but hangs indefinitely on the line using (Python.GIL()) on subsequent function calls.

• What commands did you run to trigger this issue? If you can provide a Minimal, Complete, and Verifiable example this will help us understand the issue.

        public class YoloV5
    {
        private static dynamic torch;
        private static dynamic model;
        public static List<YoloV5InferenceResult> DetectObjects(string url)
        {
            if (Runtime.PythonDLL == null)
                Runtime.PythonDLL = @"C:\Python38\python38.dll";
            
            //BUG When this function is called multiple times, it hangs on the line below.  Doesn't happen when DetectObjects is called quickly in succession, but it has been observed when there is a significant wait between calls.  
            using (Py.GIL())
            {
                if (torch == null)
                    torch = Py.Import("torch");
                if (model == null)
                    model = torch.hub.load(@"C:\Users\david\repos\yolo\yolov5\", "custom", path: @"C:\Users\david\repos\yolo\yolov5\yolov5x.pt", source:"local");
                
                dynamic img = url;
                dynamic results = model(img);
                results.show();
                dynamic resultsJson = results.pandas().xyxy[0].to_json(orient: "records");
                var serializerOptions = new JsonSerializerOptions();
                serializerOptions.PropertyNameCaseInsensitive = true;
                return JsonSerializer.Deserialize<List<YoloV5InferenceResult>>(resultsJson.ToString(), serializerOptions);   
            }
        }
    }

• If there was a crash, please include the traceback here.

Just hangs.

[matthiasburger]

@lostmsu yeah I understand.

my question is more, this

PythonEngine.Initialize();
var m_threadState = PythonEngine.BeginAllowThreads();

has to be called everytime before an action is executed, say before the using (Py.Gil()) (so I can call EndAllowThreads(threadState) and Shutdown() after execution)? Or only at startup, for initialization? You see, I read the documentation in the wiki, but it’s not that well documented (and in gitter nobody answers)

[SpicyCatGames]

this is from my personal experience for anyone experiencing the issue same as @matthiasburger . For me, it was hanging up in second function call, sometimes the third or fourth or fifth but eventually it was hanging up.

This was my code originally:

private static void RunPythonCode()
{
	// Set the path to the Python directory
	string pythonPath = Path.Combine(
		System.AppDomain.CurrentDomain.BaseDirectory,
		"Python"
	);
	string pythonScriptsPath = Path.Combine(
		System.AppDomain.CurrentDomain.BaseDirectory,
		"PythonScripts"
	);

	Environment.SetEnvironmentVariable("PATH", pythonPath + ";" + Environment.GetEnvironmentVariable("PATH"));
	Environment.SetEnvironmentVariable("PYTHONHOME", pythonPath);
	Environment.SetEnvironmentVariable("PYTHONNET_PYDLL", "python311.dll");
	Environment.SetEnvironmentVariable("PYTHONPATH", pythonScriptsPath);

	// Initialize the Python runtime
	PythonEngine.Initialize();

	// Run the Python script
	using (Py.GIL())
	{
		// Add the path to your module to the sys.path list
		dynamic sys = Py.Import("sys");
		sys.path.append(pythonScriptsPath);

		dynamic module = Py.Import("moduleName");
		module.pythonFunction();
	}
			
	PythonEngine.Shutdown();
}

The changes I made are, calling the initialize only once, using that same runtime and only calling shutdown when I don’t need to execute any more python code (which for me is never) and calling BeginAllowThreads(). Also, the stuff with sys has to be done only once.

// Set the path to the Python directory
string pythonPath = Path.Combine(
	System.AppDomain.CurrentDomain.BaseDirectory,
	"Python"
);
string pythonScriptsPath = Path.Combine(
	System.AppDomain.CurrentDomain.BaseDirectory,
	"PythonScripts"
);

// runtime_initialize is a field
if (!runtime_initialized)
{
	Environment.SetEnvironmentVariable("PATH", pythonPath + ";" + Environment.GetEnvironmentVariable("PATH"));
	Environment.SetEnvironmentVariable("PYTHONHOME", pythonPath);
	Environment.SetEnvironmentVariable("PYTHONNET_PYDLL", "python311.dll");
	Environment.SetEnvironmentVariable("PYTHONPATH", pythonScriptsPath);

	// Initialize the Python runtime
	PythonEngine.Initialize();
	PythonEngine.BeginAllowThreads();
}

// Run the Python script
using (Py.GIL())
{
	if (!runtime_initialized)
	{
		// Add the path to your module to the sys.path list
		dynamic sys = Py.Import("sys");
		sys.path.append(pythonScriptsPath);
		runtime_initialized = true;
	}

	dynamic module = Py.Import("moduleName");
	module.pythonFunction();
}