question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[QUESTION]: Save model at each epoch

See original GitHub issue

What are you trying to do? I am trying to save the model at certain number of epochs.

Previous attempts I could not find any argument in the documentation to accomplish it.

What would be a way to do it? Inspecting the structure of the final directory after training contains the final .pt.

Thanks.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
shihchenglicommented, Sep 22, 2022
        # Save each epoch
        epoch_save_dir = os.path.join(save_dir, 'epochs', f'{epoch}')
        makedirs(epoch_save_dir)
        save_checkpoint(os.path.join(epoch_save_dir, MODEL_FILE_NAME), model, scaler,
                        features_scaler, atom_descriptor_scaler, bond_feature_scaler, args)

@muammar If you are still interested in saving model of each epoch, you can add the above code at L310 in train/run_training.py.

0reactions
shihchenglicommented, Sep 23, 2022

@muammar Thanks for this suggestion. We think it doesn’t hurt to have this option in Chemprop. Actually, one of the developers has already done this before in a branch. If you are interested in adding this feature and making a PR, that would be great!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Saving your weights for each epoch — Keras callbacks
An interesting approach to solving this problem is to save your weights for each epoch or always save the best result, but how...
Read more >
Save Keras model at specific epochs - python - Stack Overflow
What I am trying to do is save the model after some specific epochs are done. Let's say for example, after epoch =...
Read more >
Keras Callback example for saving a model after every epoch?
It always saves the model every freq epochs and at the end of the training. Share.
Read more >
Keras Callbacks and How to Save Your Model from Overtraining
One cool trick for saving multiple times is to append the epoch number and/or current metric you are monitoring to the filepath, since...
Read more >
ModelCheckpoint - Keras
save_freq: 'epoch' or integer. When using 'epoch' , the callback saves the model after each epoch. When using integer, the callback saves the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found