Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Dependency parsing head for pretrained models

See original GitHub issue

🚀 Feature request

Add a new classification head for pretrained models, for dependency parsing.

Motivation

Current heads, such as AutoModelForTokenClassification, does not work well for finetuning a pretrained model for dependency parsing. There are now such heads available, so adding such would make it a lot easier to finetune models for such tasks. A PyTorch implementation of that paper (and others) can be found in this repo.

Your contribution

I could assist with a PR, but for now I’d like to start the discussion to see if this is something that the HF team and others would be interested in.

Issue Analytics

State:
Created 2 years ago
Reactions:1
Comments:8 (8 by maintainers)

Top GitHub Comments

1reaction

saattrupdancommented, Nov 9, 2021

@huberemanuel Amazing, great stuff! With the internals in place, it’s hopefully doable to tweak it in (roughly) the way I did. Thanks for all the work you’re putting into it, and would love to see your code when you’re ready to share it! 😊

1reaction

huberemanuelcommented, Oct 25, 2021

Thanks! I will take a look into it and see how can I contribute to your initiative.

Top Results From Across the Web

Dependency Parsing as Head Selection - ACL Anthology

In this work, we propose a simple neural network-based model which learns to select the head for each word in a sentence without...

Dependency parsing of biomedical text with BERT

Syntactic analysis, or parsing, is a key task in natural language processing and a required component for many text mining approaches.

Dependency Parsing | Papers With Code

This paper addressed the problem of structured sentiment analysis using a bi-affine semantic dependency parser, large pre-trained language models, and publicly ...

Dependency Parsing

Usually, dependencies form a tree (single-head, connected, acyclic) ... (De Marneffe et al, 2006): Generating typed dependency parses.

Typed Dependency Parsing for English

DescriptionTyped Dependency parser, trained on the on the CONLL dataset. ... between “head” words and words, which modify those heads.