[BUG] setuptools 62.1.0 breaks project that requires fortran sources (Nonexistent include directory)

setuptools version

62.1.0

Python version

3.10

OS

Linux, macOS

Additional environment information

I am using the version supplied by conda-forge

Description

We have a project that requires compiling fortran source files, and everything has worked fine with setuptools until version 62.1.0 (62.0.0, and all other lower versions, work!). The problem exists using macos and linux with github actions, as well as on my own macos machine. I re-emphasize, that this is a problem ONLY with 62.1.0.

I will admit that I have no idea how to debug this problem: in the build process, all of the source and module files compile, but at the very end when we try to compile the python wrapper, it complains about a non-existent include directory, which prevents the source file from finding a pre-compiled module.

INFO: gfortran:f90: src/PythonWrapper.f95
      f951: Warning: Nonexistent include directory ‘build/temp.linux-x86_64-3.8’ [-Wmissing-include-dirs]
      src/PythonWrapper.f95:2:12:
      
          2 |         use shtools, only: PlmBar
            |            1
      Fatal Error: Cannot open module file ‘shtools.mod’ for reading at (1): No such file or directory
      compilation terminated.

Expected behavior

compile with no errors, as it has been doing for the past several years…

How to Reproduce

Try to install the pyshtools project: https://github.com/SHTOOLS/SHTOOLS using pip install -v -e .

Output

Please see the github actions output for this run:

https://github.com/SHTOOLS/SHTOOLS/runs/5987783398?check_suite_focus=true

[henryiii]

This probably comes from this StackOverflow question:

https://stackoverflow.com/questions/14320220/testing-python-c-libraries-get-build-path

There’s a big warning there “This is an internal detail of setuptools (based on the bundled version of distutils), and so is not guaranteed to be stable.” even before I edited it to include the current Setuptools. The internal build directory is a private detail and building the name yourself is going to be prone to breakage.

Look at the edited response to see how you can build this (and there’s also an answer that avoids that using build_lib from build, if that’s accessible).

[henryiii]

This hardcoded assumption about the build directory is now wrong:

https://github.com/SHTOOLS/SHTOOLS/blob/8d83c42d1313d5624c4db8c2e57300c5d819834e/setup.py#L177