Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Reformer Encoder Decoder Architecture

See original GitHub issue

There seems to be a minor error in the documentation. In the example, enc_max_seq_len has been set to DE_SEQ_LEN and the dec_max_seq_len is set to EN_SEQ_LEN .

Issue Analytics

State:
Created 3 years ago
Comments:9 (4 by maintainers)

Top GitHub Comments

1reaction

AliOskooeiTRcommented, Jun 20, 2020

@lucidrains Thanks for bringing it to my attention. I will definitely try it out!

1reaction

lucidrainscommented, Jun 20, 2020

@AliOskooeiTR Yes, Routing Transformer (and all the variants of Transformers I implement), are meant to stretch attention as far and wide as possible!

Top Results From Across the Web

Reformer, the efficient Transformer, in Pytorch - GitHub

By popular demand, I have coded up a wrapper that removes a lot of the manual work in writing up a generic Reformer...

Reformer - Hugging Face

It is used to instantiate a Reformer model according to the specified arguments, defining the model architecture. Instantiating a configuration with the ...

Illustrating the Reformer. 🚊 ️ The efficient Transformer

Recently, Google introduced the Reformer architecture, a Transformer model designed to efficiently handle processing very long sequences of ...

Named Entity Recognition: A Brief Overview and ...

General Deep Learning Architecture for NER ... Reformer(): Creates a Reversible Transformer encoder-decoder model. tl.

Solving the Bottleneck of Transformer model | by Cheng He

First, let's recap on the transformer architecture: Transformer with a stack of 2 encoders and decoders, source. The Problem of Transformer:.