BUG: Building scipy from source on Windows Subsystem for Linux fails

Hi,

Kudos to all contributors for the great work on this repo!

I would also like to contribute, but to do that I’d like to be able to build from source on my Windows Subsystem for Linux (Ubuntu). So far, unfortunately, there has been no success.

The following command:

(scipydev) runner@Machine:/mnt/c/Users/runner/git/scipy$ python3 -v setup.py build 1>build.log 2>build_err.log

seems to do well until it attempts to build extension “scipy.integrate.vode” sources yielding the following errors (at the end of file build.log).

And this is how the build directory looks like after the build command above fails:

(scipydev) runner@Machine:/mnt/c/Users/runner/git/scipy$ tree build
build
└── src.linux-x86_64-3.8
    ├── build
    │   └── src.linux-x86_64-3.8
    │       └── scipy
    │           └── integrate
    │               └── fortranobject.c
    └── scipy
        ├── __config__.py
        └── integrate
            ├── vode-f2pywrappers.f
            └── vodemodule.c

7 directories, 4 files

I’m running the above in a conda virtual environment scipydev. All package requirements for the build have also been satisfied as specified by the scipy source documentation.

My windows version is version 10. WSL is version 2. Other information:

uname -a
Linux VAIO 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux  

Any help will be greatly appreciated. Thanks!

Reproducing Code Example

(scipydev) runner@Machine:/mnt/c/Users/runner/git/scipy$ python3 -v setup.py build 1>build.log 2>build_err.log

Error message

The build.log file yields (at the end):

building extension "scipy.integrate._odepack" sources
building extension "scipy.integrate.vode" sources
creating build/src.linux-x86_64-3.8/scipy/integrate
f2py options: []
f2py: scipy/integrate/vode.pyf
Reading fortran codes...
	Reading file 'scipy/integrate/vode.pyf' (format:free)
Post-processing...
	Block: dvode__user__routines
		Block: dvode_user_interface
			Block: f
			Block: jac
	Block: zvode__user__routines
		Block: zvode_user_interface
			Block: f
			Block: jac
	Block: vode
			Block: dvode
In: scipy/integrate/vode.pyf:vode:unknown_interface:dvode
get_useparameters: no module dvode__user__routines info used by dvode
			Block: zvode
In: scipy/integrate/vode.pyf:vode:unknown_interface:zvode
get_useparameters: no module zvode__user__routines info used by zvode
Post-processing (stage 2)...
Building modules...
	Constructing call-back function "cb_f_in_dvode__user__routines"
	  def f(t,y): return ydot
	Constructing call-back function "cb_jac_in_dvode__user__routines"
	  def jac(t,y): return jac
	Constructing call-back function "cb_f_in_zvode__user__routines"
	  def f(t,y): return ydot
	Constructing call-back function "cb_jac_in_zvode__user__routines"
	  def jac(t,y): return jac
	Building module "vode"...
		Constructing wrapper function "dvode"...
warning: callstatement is defined without callprotoargument
getarrdims:warning: assumed shape array, using 0 instead of '*'
getarrdims:warning: assumed shape array, using 0 instead of '*'
		  y,t,istate = dvode(f,jac,y,t,tout,rtol,atol,itask,istate,rwork,iwork,mf,[f_extra_args,jac_extra_args,overwrite_y])
		Constructing wrapper function "zvode"...
warning: callstatement is defined without callprotoargument
getarrdims:warning: assumed shape array, using 0 instead of '*'
getarrdims:warning: assumed shape array, using 0 instead of '*'
		  y,t,istate = zvode(f,jac,y,t,tout,rtol,atol,itask,istate,zwork,rwork,iwork,mf,[f_extra_args,jac_extra_args,overwrite_y])
		Constructing COMMON block support for "types"...
		  intvar
	Wrote C/API module "vode" to file "build/src.linux-x86_64-3.8/scipy/integrate/vodemodule.c"
	Fortran 77 wrappers are saved to "build/src.linux-x86_64-3.8/scipy/integrate/vode-f2pywrappers.f"
  adding 'build/src.linux-x86_64-3.8/build/src.linux-x86_64-3.8/scipy/integrate/fortranobject.c' to sources.
  adding 'build/src.linux-x86_64-3.8/build/src.linux-x86_64-3.8/scipy/integrate' to include_dirs.
creating build/src.linux-x86_64-3.8/build
creating build/src.linux-x86_64-3.8/build/src.linux-x86_64-3.8
creating build/src.linux-x86_64-3.8/build/src.linux-x86_64-3.8/scipy
creating build/src.linux-x86_64-3.8/build/src.linux-x86_64-3.8/scipy/integrate
copying /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c -> build/src.linux-x86_64-3.8/build/src.linux-x86_64-3.8/scipy/integrate




The build_err.log file, at the same time yields at the end:


import 'numpy.f2py.capi_maps' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f1455ac40>
# /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/common_rules.cpython-38.pyc matches /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/common_rules.py
# code object from '/home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/common_rules.cpython-38.pyc'
# /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/func2subr.cpython-38.pyc matches /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/func2subr.py
# code object from '/home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/func2subr.cpython-38.pyc'
import 'numpy.f2py.func2subr' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f144f4970>
import 'numpy.f2py.common_rules' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f145e1f70>
# /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/use_rules.cpython-38.pyc matches /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/use_rules.py
# code object from '/home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/use_rules.cpython-38.pyc'
import 'numpy.f2py.use_rules' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f144f4a90>
# /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/f90mod_rules.cpython-38.pyc matches /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/f90mod_rules.py
# code object from '/home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/f90mod_rules.cpython-38.pyc'
import 'numpy.f2py.f90mod_rules' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f144f4be0>
import 'numpy.f2py.rules' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f1459e2e0>
import 'numpy.f2py.f2py2e' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f1459e7f0>
# /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/diagnose.cpython-38.pyc matches /home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/diagnose.py
# code object from '/home/runner/anaconda3/envs/scipydev/lib/python3.8/site-packages/numpy/f2py/__pycache__/diagnose.cpython-38.pyc'
import 'numpy.f2py.diagnose' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f145320a0>
import 'numpy.f2py' # <_frozen_importlib_external.SourceFileLoader object at 0x7f9f1459e3a0>
append_needs: unknown need 'int'
append_needs: unknown need 'double'
append_needs: unknown need 'double'
append_needs: unknown need 'double'
append_needs: unknown need 'double'
append_needs: unknown need 'int'
append_needs: unknown need 'int'
append_needs: unknown need 'double'
append_needs: unknown need 'double'
append_needs: unknown need 'int'
append_needs: unknown need 'int'
append_needs: unknown need 'double'
append_needs: unknown need 'int'
append_needs: unknown need 'double'
append_needs: unknown need 'int'
append_needs: unknown need 'int'
append_needs: unknown need 'double'
append_needs: unknown need 'double'
append_needs: unknown need 'int'
append_needs: unknown need 'int'
append_needs: unknown need 'double'
append_needs: unknown need 'int'
append_needs: unknown need 'int'
append_needs: unknown need 'int'
append_needs: unknown need 'double'
append_needs: unknown need 'int'
error: [Errno 1] Operation not permitted
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2

### SciPy/NumPy/Python version information

(scipydev) runner@VAIO:/mnt/c/Users/runner/git/scipy$ python3 -c "import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)" Traceback (most recent call last):   File "<string>", line 1, in <module>   File "/mnt/c/Users/runner/git/scipy/scipy/__init__.py", line 153, in <module>     from scipy._lib._ccallback import LowLevelCallable   File "/mnt/c/Users/runner/git/scipy/scipy/_lib/_ccallback.py", line 1, in <module>     from . import _ccallback_c ImportError: cannot import name '_ccallback_c' from 'scipy._lib' (/mnt/c/Users/runner/git/scipy/scipy/_lib/__init__.py)

[ParticularMiner]

Sure. No problem!

[tupui]

@tupui @serge-sans-paille @rgommers @mdhaber

Many many thanks! After cleaning out the generated and untracked repo files my build went through on WSL. I guess that enabled pythran (the unpatched version) to regenerate the cpp files with forward slashes in include header file paths as it recognized WSL to be Linux-based.

Glad to hear! If there is no issue anymore. Please close the issue 😃 (IMO it’s not really a SciPy issue since starting from scratch it works. I do agree that it’s confusing though.)

Thanks for going through the logs and helping us identify this. As you can see, Serge will mitigate this upstream.