Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

No positional arguments for f2py callback

See original GitHub issue

I am trying to override SLICOT’s calls to LAPACK XERBLA in order to raise a Slycot specific exception: https://github.com/python-control/Slycot/pull/127

Unfortunately I am having issues with passing required positional arguments to the callback function.

The SLICOT routines call XERBLA(SRNAME, INFO), which should call the user provided Python function raise_xerbla(srname, info). But that function complains about not enough provided positional arguments. In fact, the argument list is empty.

Any tips what is wrong here?

>>> numpy.__version__
'1.18.4'

Issue Analytics

State:
Created 3 years ago
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

bnavigatorcommented, May 19, 2020

Hi,

sure! As you may have seen, we went another route without the callback. However it’s maybe worth fixing if this is a bug in f2py.

The attribute error means, you didn’t import the development version of Slycot with the xerbla_raises function and the sb10fd wrapper present, but likely an installed Slycot 0.3.5.

Here is a minimal example without involving Slycot at all: https://github.com/bnavigator/f2py-minimal-cb

greiner@greinerT450s:~/src/f2py-minimal-cb/ > f2py -c -m minimal_fortran minimal_fortran.pyf XERBLA.f FORTRAN_CALLER.f
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "minimal_fortran" sources
creating /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8
f2py options: []
f2py: minimal_fortran.pyf
Reading fortran codes...
        Reading file 'minimal_fortran.pyf' (format:free)
Line #19 in minimal_fortran.pyf:"            intent(callback) python_callback "
        analyzeline: missing __user__ module (could be nothing)
Line #19 in minimal_fortran.pyf:"            intent(callback) python_callback "
        analyzeline: appending intent(callback) python_callback to xerbla arguments
Post-processing...
        Block: xerbla__user__routines
                Block: xerbla_user_interface
                        Block: python_callback
        Block: minimal_fortran
                        Block: xerbla
In: minimal_fortran.pyf:minimal_fortran:unknown_interface:xerbla
get_useparameters: no module xerbla__user__routines info used by xerbla
                        Block: fortran_caller
Post-processing (stage 2)...
Building modules...
        Constructing call-back function "cb_python_callback_in_xerbla__user__routines"
          def python_callback(srname,info): return 
        Building module "minimal_fortran"...
                Constructing wrapper function "xerbla"...
                  xerbla(srname,info,python_callback,[python_callback_extra_args])
                Constructing wrapper function "fortran_caller"...
                  fortran_caller(a1)
        Wrote C/API module "minimal_fortran" to file "/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/minimal_fortranmodule.c"
  adding '/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/fortranobject.c' to sources.
  adding '/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8' to include_dirs.
copying /usr/lib64/python3.8/site-packages/numpy/f2py/src/fortranobject.c -> /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8
copying /usr/lib64/python3.8/site-packages/numpy/f2py/src/fortranobject.h -> /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8
build_src: building npy-pkg config files
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
get_default_fcompiler: matching types: '['gnu95', 'intel', 'lahey', 'pg', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor']'
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using build_ext
building 'minimal_fortran' extension
compiling C sources
C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -fPIC

creating /tmp/tmpr4u7s_0z/tmp
creating /tmp/tmpr4u7s_0z/tmp/tmpr4u7s_0z
creating /tmp/tmpr4u7s_0z/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8
compile options: '-I/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8 -I/usr/lib64/python3.8/site-packages/numpy/core/include -I/usr/include/python3.8 -c'
gcc: /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/minimal_fortranmodule.c
gcc: /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/fortranobject.c
In file included from /usr/lib64/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1832,
                 from /usr/lib64/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /usr/lib64/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/fortranobject.h:13,
                 from /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/minimal_fortranmodule.c:16:
/usr/lib64/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
In file included from /usr/lib64/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1832,
                 from /usr/lib64/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /usr/lib64/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/fortranobject.h:13,
                 from /tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/fortranobject.c:2:
/usr/lib64/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/minimal_fortranmodule.c: In function ‘python_callback_’:
/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/minimal_fortranmodule.c:413:7: warning: variable ‘capi_j’ set but not used [-Wunused-but-set-variable]
  413 |   int capi_j,capi_i = 0;
      |       ^~~~~~
At top level:
/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/minimal_fortranmodule.c:149:12: warning: ‘f2py_size’ defined but not used [-Wunused-function]
  149 | static int f2py_size(PyArrayObject* var, ...)
      |            ^~~~~~~~~
compiling Fortran sources
Fortran f77 compiler: /usr/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran f90 compiler: /usr/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran fix compiler: /usr/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
compile options: '-I/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8 -I/usr/lib64/python3.8/site-packages/numpy/core/include -I/usr/include/python3.8 -c'
gfortran:f77: XERBLA.f
gfortran:f77: FORTRAN_CALLER.f
/usr/bin/gfortran -Wall -g -Wall -g -shared /tmp/tmpr4u7s_0z/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/minimal_fortranmodule.o /tmp/tmpr4u7s_0z/tmp/tmpr4u7s_0z/src.linux-x86_64-3.8/fortranobject.o /tmp/tmpr4u7s_0z/XERBLA.o /tmp/tmpr4u7s_0z/FORTRAN_CALLER.o -L/usr/lib64/gcc/x86_64-suse-linux/9 -L/usr/lib64/gcc/x86_64-suse-linux/9 -L/usr/lib64 -lgfortran -o ./minimal_fortran.cpython-38-x86_64-linux-gnu.so
Removing build directory /tmp/tmpr4u7s_0z
greiner@greinerT450s:~/src/f2py-minimal-cb/ > python3 python_script.py 
capi_return is NULL
Call-back cb_python_callback_in_xerbla__user__routines failed.
Traceback (most recent call last):
  File "python_script.py", line 25, in <module>
    minimal_fortran.fortran_caller(1)
TypeError: python_callback() missing 2 required positional arguments: 'caller' and 'arg'
greiner@greinerT450s:~/src/f2py-minimal-cb/ >

0reactions

bnavigatorcommented, Jul 14, 2020

Hi @melissawm, sorry for the late response. Thank you very much for looking into this. As mentioned before, be already went another route without calling the callback.

I can imagine that it would be great to have the functionality initially described, though.

If I understand correctly, the solution you describe requires the additition of the callback signature to all Fortran routines calling the intermediate routine. So overriding an existing generic Fortran routine with one using a Python callback is not possible without adjusting the code for the complete library. Would it be possible to extend f2py so that it traverses routine signatures and injects the dependency automatically? Bonus: What about Fortran objects, which are compiled and used by .pyf referenced routines but are not in the interface themselves?