DLL resolution in Python 3.8 on Windows has changed

The resolution of DLLs has changed in Python 3.8 for Windows.

New in version 3.8: Previous versions of CPython would resolve DLLs using the default behavior for the current process. This led to inconsistencies, such as only sometimes searching PATH or the current working directory, and OS functions such as AddDllDirectory having no effect.

DLL dependencies for extension modules and DLLs loaded with ctypes on Windows are now resolved more securely. Only the system paths, the directory containing the DLL or PYD file, and directories added with add_dll_directory() are searched for load-time dependencies. Specifically, PATH and the current working directory are no longer used, and modifications to these will no longer have any effect on normal DLL resolution. If your application relies on these mechanisms, you should check for add_dll_directory() and if it exists, use it to add your DLLs directory while loading your library. Note that Windows 7 users will need to ensure that Windows Update KB2533623 has been installed (this is also verified by the installer). (Contributed by Steve Dower in bpo-36085.)

• https://bugs.python.org/issue36085
• https://docs.python.org/3/whatsnew/3.8.html#bpo-36085-whatsnew
• https://docs.python.org/3/library/os.html#os.add_dll_directory

As a result the Appveyor job for 3.8 is failing:

https://ci.appveyor.com/project/sgillies/fiona/branch/master/job/5e9p53tga9ono4j3

python -c "import fiona"
python : Traceback (most recent call last):
At line:1 char:1
+ python -c "import fiona"
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Traceback (most recent call last)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
  File "<string>", line 1, in <module>
  File "C:\Python38-x64\Lib\site-packages\fiona\__init__.py", line 85, in <module>
    from fiona.collection import BytesCollection, Collection
  File "C:\Python38-x64\Lib\site-packages\fiona\collection.py", line 9, in <module>
    from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
ImportError: DLL load failed while importing ogrext: The specified module could not be found.
Command executed with exception:   File "<string>", line 1, in <module>
  File "C:\Python38-x64\Lib\site-packages\fiona\__init__.py", line 85, in <module>
    from fiona.collection import BytesCollection, Collection
  File "C:\Python38-x64\Lib\site-packages\fiona\collection.py", line 9, in <module>
    from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
ImportError: DLL load failed while importing ogrext: The specified module could not be found.

I’m not sure what the correct thing to do here is. For wheels on Windows it’s easy to call add_dll_directory with the correct path. But I’m not sure what to do for users not using a wheel - how could we know which path to add? I’d be interested to see how other libraries have handled this.

[gtmaskall]

@achapkowski - were you seeing this (with Python 3.6.8) in either ipython or a jupyter notebook? I’ve just had a very similar issue with python 3.6.11 that is resolved by upgrading to python 3.6.12. The weird thing is that it only happened when using either ipython or jupyter notebooks, and not the straight python repl. I documented my research in this SO post.

@rbuffat , @snorfalorpagus - do the tests cover imports from ipython?

[GuYith]

I found the same problem when I tried to “import fiona” in Jupyter Notebook.

As a matter of fact, I had no problem importing in Jupyter Notebook when I had configured the environment at the beginning. However, when I tried to use it again a few days later, the same error occurred. And I found, the import outside of Jupyter Notebook is correct.

I support the problem is not related to fiona itself.

I tried removing the kernel which include fiona, and installed it in Jupyter Notebook again. The error disappeared. Although I don’t know why.