In dependent project, ak.layout fails to convert to ak::Content. Update pybind11!

I’ve been working on some fastjet bindings with an awkward1 interface based on pybind11 (I’m aware of pyjet and the SWIG bindings, but for various reasons, this was a better approach for me. I’m not linking them here because they’re early and ugly and not really ready for anyone else, but the repo can be found on my profile if it’s somehow helpful). I have an early version working on macOS, but when I moved to linux (ubuntu 18.04 for what it’s worth, but I’m working in a virtualenv), it throws a TypeError whenever I try to pass an array layout to c++. For a trivial function which just returns what it receives, on linux I get

Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import awkward1 as ak
>>> import pybind11_awkward as pba
>>> pba.awkward_test(ak.Array([1,2,3]).layout)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: awkward_test(): incompatible function arguments. The following argument types are supported:
    1. (arr: awkward::Content) -> awkward::Content

Invoked with: <NumpyArray format="l" shape="3" data="1 2 3" at="0x000001be97b0"/>

On macOS, I get:

Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import awkward1 as ak
>>> import pybind11_awkward as pba
>>> pba.awkward_test(ak.Array([1,2,3]).layout)
In function
<NumpyArray format="l" shape="3" data="1 2 3" at="0x7ff17b283a00"/>

Other functions seem to work fine - it’s apparently something about passing awkward arrays. I’m using awkward 0.3.1.

I’m a bit stuck on how to proceed. I recognize there’s a huge phase space where I could have gone wrong, so I made a quick repository that reproduces the issue, and I tried to keep as much consistent as possible. It’s available here. You can install it with poetry install, and then run test_simple.py. (In principle, it should work with pip using PEP 517, but my build script isn’t quite right, sorry). I’m not 100% sure that this is actually an awkward1 issue (there are still some differences in the setups), but it seems like a reasonable place to start.

As always, help or suggestions are greatly appreciated! Thanks!

[jpivarski]

Oh, okay—a platform-dependent error (worked in CI, failed on your system). I’m glad the new pybind11 fixes that because it would be tough to debug.

The next version will likely come out when Python 3.9 can be tested, which will be as soon as it’s added to Azure’s pipelines. (c.f. microsoft/azure-pipelines-tool-lib#77).

[raymondEhlers]

Sorry, I think I wasn’t super clear on this point: this issue was triggered just by passing arrays from python -> cpp. It also occurred trying to compile and run dependent.cpp on our cluster. So I don’t see any obvious additional functionality to add to dependent.cpp that will trigger this issue. Which makes this especially puzzling to me. In any case, if I think of a test as I keep developing my code, I’ll definitely open a PR.

Sounds good - I look forward to the next release!

Thanks again to you and @veprbl for all of the help!