question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Cython dependency handling in .pxd and .pxi files

See original GitHub issue

Describe the bug

Cython has a notion of implementation source files (.pyx), declaration files (.pxd) and literal include files (.pxi). The distinction is similar to .c and .h files in C. The current syntax for tracking changes in declaration and include files is clumsy, especially for the cython->c++ transpile.

To Reproduce

In the current master branch, to specify a cython->c++ transile for a source with a .pyx and a .pxd file, depend_files argument to custom_target works (thanks to @rgommers for pointing this out). Here is an example where the implementation is split between observable.pxd and observable.pyx (https://github.com/ev-br/mc_lib/blob/master/mc_lib/meson.build):

# compile .cpp for the `observable` module
_observable_cpp = custom_target('observable_cpp',
  output : 'observable.cpp',
  input : 'observable.pyx',
  depend_files: 'observable.pxd',     # <---- this
  command : [cython, '--cplus', '-3', '--fast-fail', '@INPUT@', '-o', '@OUTPUT@']
)

# generate .so for observable
py3.extension_module(
  'observable', _observable_cpp,
  include_directories: inc_np,
  dependencies : [py3_dep],
  install: true,
  subdir: 'mc_lib'
)

With gh-9017, the syntax is greatly simplified:

# generate .so for the `observable` extension
py3.extension_module(
  'observable', 'observable.pyx',
  include_directories: inc_np,
  dependencies : py3_dep,
  override_options : ['cython_language=cpp'],   # <-----  no need for a custom_target, yay!
  install: true,
  subdir: 'mc_lib',
)

However, this does not track changes in observable.pxd. To reproduce the problem: compile once, introduce a syntax error into observable.pxd, try recompiling ( I did $ meson install -C build, but a direct ninja invocation would do) — ninja does not see a syntax error:

ninja: Entering directory `/home/br/sweethome/proj/mc_lib/builddir'
ninja: no work to do.

Expected behavior

Expected behavior is that a change in a pxd or pxi file triggers a rebuild of a dependent extension module: similar to what happens for the .c / .h pairs. Either automatically, or via a simple meson.build syntax to declare a dependency.

system parameters

Native build on ubuntu focal, python 3.8.10, $ meson --version 0.59.99 $ ninja --version 1.10.2.git.kitware.jobserver-1

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
ev-brcommented, Jan 28, 2022

Depfile support has landed in the cython main branch.

Also 0.29.x branch now. (0.29.27, to be specific)

0reactions
ev-brcommented, Jan 14, 2022

Depfile support has landed in the cython main branch.

Read more comments on GitHub >

github_iconTop Results From Across the Web

pxd files — Cython 3.0.0a11 documentation
pxd files ¶ · They can be used for sharing external C declarations. · They can contain functions which are well suited for...
Read more >
1445411 – Review Request: python-cysignals - Interrupt and signal ...
For example, code like the following cannot be interrupted in Cython: while True: pass ... Also, is it really necessary to install the...
Read more >
Moving SciPy to the Meson build system | Quansight Labs
SciPy contains C, C++, Fortran and Cython code, exports C and Cython ... and .pxi files at build time, and the .pxd files...
Read more >
How to use a cython definition file (pxd) without making it an ...
py files. This includes making an Extension for common.pyx in your setup.py file. You may want to try using cythonize to handle the...
Read more >
Reported by - Sage Trac - SageMath
Install sage headers and auxiliary files (.h/.pxd/.pxi files) ... Currently, Sage doesn't install any files other than .py and compiled cython code.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found