train model?

Would it be possible to train a model on our own data leveraging pretrained BERT models from transformerslibrary for instance? I don’t see anything related to the training part (on the Pytorch part in particular)

[pommedeterresautee]

Thanks. I close the issue, however I think that sharing the training part source code would be really great (even if easy to adapt). Cho code is old and TF based, TF is changing all the time, it requires an analysis to be sure to have the same hyper parameters, etc.

[pertschuk]

We use reuse of raw logits as in Cho paper, and model code based on huggingface’s transformers library.

The training can be benefited from gradually increasing the number of positive examples to avoid overfitting (i forget the name of this technique off the top of my head…)