from_dlpack in spacy-transformers

The test suite for spacy-transformers breaks in my local environment since PR https://github.com/explosion/thinc/pull/686

numpy                             1.23.1
spacy                             3.4.1        (current master)
spacy-transformers                1.1.2        (current master)
thinc                             8.1.0        (current master)
torch                             1.9.0+cu111

e.g. when I run test_model_sequence_classification, it crashes at https://github.com/explosion/thinc/blob/master/thinc/util.py#L365:

>           torch_tensor = torch.utils.dlpack.from_dlpack(xp_tensor)
E           RuntimeError: from_dlpack received an invalid capsule. Note that DLTensor capsules can be consumed only once, so you might have already constructed a tensor from it once.

But, when I look at the code behind from_dlpack and run this directly (having confirmed my system doesn’t hit the if statement checking the device):

dlpack = xp_tensor.__dlpack__()
torch_tensor = torch._C._from_dlpack(dlpack)

Then the test flies through.

Can anyone reproduce the failing tests on the spacy-transformers test suite with a relatively new numpy?

[polm]

This also came up in the coref PR in spacy-experimental. It was pretty confusing but upgrading the Torch version did fix it.

[svlandeg]

Ok, good points. Let’s close this and leave as-is for now. If we do get external reports, we can consider adding a warning.