Replace scatter operations in TAPAS by native PyTorch
See original GitHub issueFeature request
TAPAS (my first 🤗 contribution 😄 ) still relies on the torch_scatter library, as the model uses some scatter operations on tensors. Back then PyTorch didn’t have these operations available.
Now they have: https://pytorch.org/docs/stable/generated/torch.Tensor.scatter_.html#torch.Tensor.scatter_.
So we should replace this line by native PyTorch.
To confirm everything is working fine, one should run the following tests and make sure they pass (to be run from the root of this repository):
RUN_SLOW=yes pytest tests/models/tapas/test_modeling_tapas.py
Subsequently, all is_scatter_available
mentions can be removed from the code base.
Motivation
By replacing this, our TAPAS implementation doesn’t rely on a third-party library anymore.
Your contribution
I can look into this, but marking it as a good first/second issue.
Update: I have an attempt here: https://github.com/huggingface/transformers/compare/main...NielsRogge:transformers:fix_tapas_scatter?expand=1.
However, training tests didn’t pass due to an issue in the backward pass.
Issue Analytics
- State:
- Created 10 months ago
- Reactions:1
- Comments:8 (6 by maintainers)
Yes, thanks for reminding us 😃 @Bearnardd
Closed by #20149
@Bearnardd Thank you!
You can start from this (updated) branch/PR (which is updated with the
main
branch with a tiny fix). If you want to start from @NielsRogge branch, you have to rebase on (updated)main
first - there will be a few conflicts to resolve.https://github.com/huggingface/transformers/pull/20107 https://github.com/huggingface/transformers/tree/fix_tapas_scatter