Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pre training problem, please help me out

See original GitHub issue

Environment info

transformers version: 4.8.2
Platform: Linux-5.4.104±x86_64-with-Ubuntu-18.04-bionic
Python version: 3.7.11
PyTorch version (GPU?): 1.9.0+cu102 (False)
Tensorflow version (GPU?): 2.5.0 (False)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: no
Using distributed or parallel set-up in script?: no

Who can help

@sgugger @LysandreJik @patrickvonplaten @patil-suraj

Information

Model I am using: CodeBert

The problem arises when using:

[X ] the official example scripts: (give details below)
my own modified scripts: (give details below)

The tasks I am working on is:

an official GLUE/SQUaD task: (give the name)
my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

Using the run_mlm.py script from : https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling 2.Inserting the code on terminal: python run_mlm.py --model_name_or_path microsoft/codebert-base --train_file /content/TrainingFiles.txt --do_train --do_eval --output_dir /tmp/test-mlm
Getting this error python run_mlm.py --model_name_or_path microsoft/codebert-base --train_file /content/TrainingFiles.txt --do_train --do_eval --output_dir /tmp/test-mlm File "run_mlm.py", line 67 model_name_or_path: Optional[str] = field( ^ SyntaxError: invalid syntax

Expected behavior

I just want to further pre train my model, any help would be much appreciated

Issue Analytics

State:
Created 2 years ago
Comments:11 (5 by maintainers)

Top GitHub Comments

1reaction

NielsRoggecommented, Jul 21, 2021

Ok let me try this, I get back to you in a minute.

0reactions

mosh98commented, Aug 14, 2021

okay that makes sense, Huge thanks again Niels! 😄

Have a wonderful day!

Top Results From Across the Web

Why does pre-training help avoid the vanishing gradient ...

Your question touches on two topics: Preprocessing of the data. Initialization of weights. For this question there are already good answers: ...

issues with pretrain mBART models #2120 - GitHub

While I fine-tuning mBART pre-trained CC25 model on wmt16 en-ro dataset,it was always running into out of memory problems.Like this.

On the Importance and Applicability of Pre-Training for ...

Applying pre-training to federated learning is thought to be difficult, mainly due to privacy concerns. This paper tackles this problem by ...

Using Pre-Training Can Improve Model Robustness ... - arXiv

We show that although pre-training may not improve performance on traditional classification metrics, it improves model robust- ness and uncertainty estimates.

Why Leadership Training Fails—and What to Do About It

It's much easier for HR to point to employees' competencies as the problem and to training as the clear solution. That's a message...