Setting Virtual Environment while Embedding Python in C#

Environment

• Pythonnet version: Latest as on 7 Jan 2021 cloned from github official repo
• Python version: 3.7.8
• Operating System: Windows 10
• .NET Runtime: .Net Core 3.1

Details

P.S. - I was able to solve this issue, however since I didn’t find any answer responding to this aspect anywhere, thought it might help the community.

• I was trying to run some python codes from my dotnet core project by setting up a local python 3.7 virtual environment created using venv. I setup the compile-time constants in project properties accordingly and followed the steps mentioned here to use my virtual environment. I was trying to import numpy which was already installed in the venv, but every time I was getting a missing basic python library error (like codec etc).

• On further inspection I found that the virtual environment directory does not have these “basic” python libraries. These libraries are present in the parent python 3.7 directory (the python which was used to create the venv itself). And since the path to the parent library is alerady present in the PythonPath, they are referred from there when you run python in CMD.

• But as mentioned in your documentation, instead of appending new values to PYTHONPATH, you are changing it completely and pointing it towards just the venv directory which does not have all the python files required.

• I was finally able to run python and import the modules in venv after appending the venv path to original PYTHONPATH, instead of assigning it directly.

• To summarise, the PythonEngine.PythonPath should have the </path/to/Lib/>, </path/to/Lib/SitePackages/> of the virtual environment python directory and also of the parent python used to create the virtual env. (Another approach could be to check the paths in sys.path in the python virtual environment and ensure those values are present here too.)

• What commands did I run to trigger this issue?

    string pathToVirtualEnv = /path/to/venv/;

    Environment.SetEnvironmentVariable("PATH", pathToVirtualEnv, EnvironmentVariableTarget.Process);
    Environment.SetEnvironmentVariable("PYTHONHOME", pathToVirtualEnv, EnvironmentVariableTarget.Process);
    Environment.SetEnvironmentVariable("PYTHONPATH", $"{pathToVirtualEnv}\\Lib\\site-packages;{pathToVirtualEnv}\\Lib", EnvironmentVariableTarget.Process);
            
            
    PythonEngine.PythonHome = pathToVirtualEnv;
    PythonEngine.PythonPath = Environment.GetEnvironmentVariable("PYTHONPATH", EnvironmentVariableTarget.Process);

• One of the errors which I faced

ImportError : No module named '_ctypes'

This change to the above code finally worked for me. Note - my initial PYTHONPATH points to python path corresponding the parent python I used to create the virtual environment (i.e. Python 3.7)

PythonEngine.PythonPath = PythonEngine.PythonPath + ";"+Environment.GetEnvironmentVariable("PYTHONPATH", EnvironmentVariableTarget.Process);

[vks2]

you saved my day. thx

[screig]

How I got it working

*Python 3.8 *VS2022 *Target framework 4.7.2 *Platform Target x64 (note you must change this, you can’t leave it on ANYCPU) *Pythonnet 3.0.1 (via nuget)

    private static void setup_py_venv_003()
    {
        Runtime.PythonDLL = @"C:\Python38\python38.dll";
        var pathToVirtualEnv = Path.GetFullPath(Path.Combine(Directory.GetCurrentDirectory(), @"..\..\..\venv"));
        Console.WriteLine(pathToVirtualEnv);


        Console.WriteLine(Runtime.PythonDLL);
        Console.WriteLine(PythonEngine.Platform);
        Console.WriteLine(PythonEngine.MinSupportedVersion);
        Console.WriteLine(PythonEngine.MaxSupportedVersion);
        Console.WriteLine(PythonEngine.BuildInfo);
        Console.WriteLine(PythonEngine.PythonPath);

        string additional = $"{pathToVirtualEnv};{pathToVirtualEnv}\\Lib\\site-packages;{pathToVirtualEnv}\\Lib";
        PythonEngine.PythonPath = PythonEngine.PythonPath + ";" + additional;
        Console.WriteLine(PythonEngine.PythonPath);

        PythonEngine.Initialize();
        PythonEngine.BeginAllowThreads();
    }

So I am adding three parts on to the end of the existing python path

1. Root {pathToVirtualEnv}
2. Site Packages {pathToVirtualEnv}\Lib\site-packages
3. Lib folder {pathToVirtualEnv}\Lib

Output

C:\Users\sean\_CODE\_SMALL\PythonCore\venv
----------
C:\Python38\python38.dll
win32
4.7
4.11.2147483647.2147483647
tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50
C:\Python38\python38.zip;C:\Python38\Lib\;C:\Python38\DLLs\;C:\Users\sean\_CODE\_SMALL\PythonCore\ConsoleCshrp\bin\Debug
----------
C:\Python38\python38.zip;C:\Python38\Lib\;C:\Python38\DLLs\;C:\Users\sean\_CODE\_SMALL\PythonCore\ConsoleCshrp\bin\Debug;C:\Users\sean\_CODE\_SMALL\PythonCore\venv;C:\Users\sean\_CODE\_SMALL\PythonCore\venv\Lib\site-packages;C:\Users\sean\_CODE\_SMALL\PythonCore\venv\Lib
----------

Then it goes on to run these successfully

 private static void run_some_tests()
    {
        using (Py.GIL())
        {
            dynamic np = Py.Import("numpy");
            Console.WriteLine(np.cos(np.pi * 2));

            dynamic sin = np.sin;
            Console.WriteLine(sin(5));

            double c = (double)(np.cos(5) + sin(5));
            Console.WriteLine(c);

            dynamic a = np.array(new List<float> { 1, 2, 3 });
            Console.WriteLine(a.dtype);

            dynamic b = np.array(new List<float> { 6, 5, 4 }, dtype: np.int32);
            Console.WriteLine(b.dtype);

            Console.WriteLine(a * b);
            Console.ReadKey();
        }
    }

A bit silly, but note in your venv you obviously need to have installed numpy into your venv first.