Filesystem significant languages and build directory implementation details

Meson has a strict rule that the layout of the build directory as an implementation detail, this makes a lot of sense as it gives us (Meson) flexibility to change the layout when it makes sense (we’ve done this a number of times, including with things like shortening the names of generated directories, or adding the private dir). This is also completely fine for languages like C in which the filesystem layout is an implementation detail of the language, and can easily be fixed with include_directories(). There are however, a growing number of languages supported by Meson (Rust, Python), or with pending support in Meson (Cython, Zig), in which the filesystem layout is the import layout. This introduces real problems when trying to mix generated and static sources, as the compilers generally expect a single, unified tree.

I’d like to propose then a way to provide sources as a tree, but in an abstract way such that Meson can make configure time decisions about what it wants to do to fulfill these requirements. Specifically, I’m proposing:

executable(
  'foo',
  {
    '' : 'main.rs',
    'foo' : [generated_rs],
    'foo/bar' : ['other.rs'],
  }
)

This specifically would tell Meson that when compiling a file main.rs, it must be in a directory such that there is a subdir foo, which contains the output of generated_rs (whatever that happens to be), and foo must have a subdir bar, with other.rs. Meson could at configure time decide that all of the inputs are static (non-generated) files, and do nothing, or it could decide that since generated_rs is a custom_target, to copy/link all of the files into a new directory and compile them there.

This solves the problem that Rust, Zig, and Cython have of needing a single tree of filres, but also creates an abstraction such that Meson can continue to hide the actual layout of the build dir, as this tree could exist anywhere within the build dir.

[dcbaker]

Yeah, the idea behind the structured inputs was that you could just describe the layout to Meson and it would ensure that it exists. It actually uses (many) custom targets behind the scene to do the copies if you need them.

Also, yes, / is portable in the Meson DSL, we os.normpath them

[rgommers]

Maybe that’s not too bad indeed. The first part should then be:

_decomp_update_pyx = custom_target('_decomp_update',
  output : 'linalg/_decomp_update.pyx',
  input : 'linalg/_decomp_update.pyx.in')   # or some variant (is `/` portable?)

It seems like this code is then in the wrong meson.build file - basically if you have a lot of Cython code that all ends up in the top-level directory rather than right next to where to .pyx files live. But maybe that’s not too much of an issue.

The alternative I thought of was to have one custom_target at the top level that simply copies over all __init__.py and all .pxd files into the build dir.