Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[readability] Consolidate prune_heads logic to PretrainedModel.

See original GitHub issue

Many models have identical implementations of prune_heads it would be nice to store that implementation as a method on PretrainedModel and reduce the redundancy.

Issue Analytics

State:
Created 3 years ago
Reactions:1
Comments:10 (2 by maintainers)

Top GitHub Comments

1reaction

sshleifercommented, May 26, 2020

Go for it!

0reactions

sshleifercommented, Jul 8, 2020

I think this is done. Happy to find new bugs if anyone is on the hunt!

Top Results From Across the Web

Models

PreTrainedModel takes care of storing the configuration of the models and handles ... resize the input embeddings,; prune heads in the self-attention heads....

Proceedings of the 2020 Conference on Empirical Methods ...

Incremental Event Detection via Knowledge Consolidation Networks ... Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained ...

Interpretable Neural Predictions with Differentiable Binary ...

It was previously used to prune heads in multi-head attention (Voita et al., 2019). Our work is more similar in spirit to Bastings...

Paper Digest: EMNLP 2020 (Main Track) Highlights

Highlight: In this paper, we propose a Knowledge Consolidation Network ... logical forms, and implementing the missing execution engines.