question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

transfer learning broken using weights from google drive

See original GitHub issue

train.py with --weights=./yolov4.weights cause an assert error under utils.load_weights If assert len(wf.read()) == 0, 'failed to read all data' is commented the training start with nan

=> STEP   63   lr: 0.000027   giou_loss:  nan   conf_loss:  nan   prob_loss:  nan   total_loss:  nan
=> STEP   64   lr: 0.000028   giou_loss:  nan   conf_loss:  nan   prob_loss:  nan   total_loss:  nan

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:2
  • Comments:12 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
bgaminicommented, May 4, 2020

I solved the problem by using pickle, but definitely this is not the correct way 😃

weights_to_pickle.py:

from core.yolov4 import YOLOv4, decode
import tensorflow as tf
import core.utils as utils
import pickle

input_layer = tf.keras.layers.Input([608, 608, 3])
feature_maps = YOLOv4(input_layer, NUM_CLASS=80)
model = tf.keras.Model(input_layer, feature_maps)
utils.load_weights(model, "./data/yolov4.weights")
weights = []
for layer in model.layers:
    weights.append(layer.get_weights())
pickle.dump(weights, open("./data/yolov4.pkl", 'wb'))

and adding this to train.py file after line 78:

elif FLAGS.weights.split(".")[len(FLAGS.weights.split(".")) - 1] == "pkl":
    weights = pickle.load(open(FLAGS.weights, 'rb'))
    num_loaded_layers = 0
    for i, w in enumerate(weights):
        try:
            model.layers[i].set_weights(w)
        except:
            num_loaded_layers = i
            break
    print("=========", num_loaded_layers ,"layers loaded successfully ==========")

Note: you can use model.layers[i].trainable = False

0reactions
hunglc007commented, Jul 10, 2020

I updated mish activation to fix nan loss issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Google Colab mount Google Drive and connect with Weights ...
Google Colab mount Google Drive and connect with Weights and Biases to track model training progress(cool TensorBoard alternative)
Read more >
Transfer Learning: Serial versus One-time Training - Wandb
The Weights & Biases platform will facilitate comparison between the training run parameters. And, if you're curious, you're of course welcome ...
Read more >
How Does Transfer learning Speeds Up Deep ... - eInfochips
This approach helps to adapt the weights from previous learning and makes training faster.
Read more >
google colaboratory, weight download (export saved models)
This worked for me !! Use PyDrive API !pip install -U -q PyDrive from pydrive.auth import GoogleAuth from pydrive.drive import GoogleDrive ...
Read more >
A Comprehensive Hands-on Guide to Transfer Learning with ...
No knowledge is retained which can be transferred from one model to another. In transfer learning, you can leverage knowledge (features, weights ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found