Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

The Document of LongT5 confilcts with and its example code of prefix

See original GitHub issue

System Info

All.

Who can help?

@patrickvonplaten

Reproduction

See https://huggingface.co/docs/transformers/main/en/model_doc/longt5

Expected behavior

In the above document, it said Unlike the T5 model, LongT5 does not use a task prefix. Furthermore, it uses a different pre-training objective inspired by the pre-training of [PegasusForConditionalGeneration].. But in the example code of LongT5ForConditionalGeneration, there is a prefix of summarize: . I am confused about how to use LongT5 in different down tasks. Could you please help? Thanks.

Issue Analytics

State:
Created a year ago
Comments:7 (3 by maintainers)

Top GitHub Comments

1reaction

patrickvonplatencommented, Aug 30, 2022

Hey @GabrielLin

That depends on how different the use cases are and what your limitations are exactly. In general, I’d say yes you should use different fine-tuned models for different tasks

0reactions

GabrielLincommented, Sep 4, 2022

@patrickvonplaten Got it. Thanks. This issue has been fixed and closed.

Top Results From Across the Web

LongT5 - Hugging Face

It's an encoder-decoder transformer pre-trained in a text-to-text denoising ... In this paper, we present a new model, called LongT5, with which we...

LongT5: Efficient Text-To-Text Transformer for Long Sequences

In this paper, we present a new model, called LongT5, with which we explore the effects of scaling both the input length and...

T5: Exploring Limits of Transfer Learning with Text ... - YouTube

trasferlearning #t5 #googleThis paper from Google introduces T5 model (Text-to-Text Transfer Transformer) and releases large scale C4 corpus ...

LongT5: Efficient Text-To-Text Transformer for ... - YouTube

t5 #transformers #nlpLongT5 explores the effect of scaling both the input length and model size of T5 at the same time with some...

math_qa | TensorFlow Datasets

A large-scale dataset of math word problems and an interpretable neural math problem ... Additional Documentation: Explore on Papers With Code north_east.