Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Code implementation - Training

See original GitHub issue

Hello,

I am trying to run the training code but I come to this error:

2020-11-02 07:36:08.658676: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Downloading: 100% 232k/232k [00:00<00:00, 318kB/s]
Downloading: 100% 442/442 [00:00<00:00, 284kB/s]
Downloading: 100% 268M/268M [00:04<00:00, 62.7MB/s]
Traceback (most recent call last):
  File "train_ditto.py", line 103, in <module>
    run_tag)
  File "Snippext_public/snippext/mixda.py", line 253, in initialize_and_train
    alpha_aug=hp.alpha_aug)
  File "Snippext_public/snippext/mixda.py", line 152, in train
    with amp.scale_loss(loss, optimizer) as scaled_loss:
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/apex/amp/handle.py", line 82, in scale_loss
    raise RuntimeError("Invoked 'with amp.scale_loss`, but internal Amp state has not been initialized.  "
RuntimeError: Invoked 'with amp.scale_loss`, but internal Amp state has not been initialized.  model, optimizer = amp.initialize(model, optimizer, opt_level=...) must be called before `with amp.scale_loss`.

Please, can you guide me to solve this issue?

Issue Analytics

State:
Created 3 years ago
Comments:9 (3 by maintainers)

Top GitHub Comments

1reaction

oi02lylcommented, Nov 2, 2020

Can you share the command that you used to run the code?
Are you using GPU or CPU?

By looking at the error msg, it seems to me that fp16 is on but GPU is not in used.

0reactions

saharyicommented, Aug 18, 2021

https://colab.research.google.com/drive/1zCg6BeCWVj62uYqoxR5rfyEG6dfGXu_2?usp=sharing&authuser=3#scrollTo=9qxLFPNvcGgH

this link don’t works for me! I get this: There was an error loading this notebook. Ensure that the file is accessible and try again. Invalid Credentials https://drive.google.com/drive/?action=locate&id=1zCg6BeCWVj62uYqoxR5rfyEG6dfGXu_2&authuser=3 please help me.

Top Results From Across the Web

CODE Training | Classes

We offer standard classes as well as custom and individual training that can be delivered on-site at your location, in our training center,...

Coding Education & Training - AHIMA

Get hands-on medical coding experience with this three-module course addressing coding practice, professional development, and career preparation. Practice ...

FREE ICD-10 Implementation Training Manual - AAPC

For a limited time, AAPC is offering our "ICD-10 Implementation Training Manual" for free (a $199 value). We want to help you prepare...

Certified Implementation Specialist - ServiceNow Training and ...

Get certified in ServiceNow implementation and elevate your talent above the rest. Find out more about our certification program today.

Michigan Uniform Energy Code Training and Implementation ...

The Michigan Uniform Energy Code Training and Implementation Program was created by SPDC. The curriculum is designed to assist home builders, architects, ...