How to evaluate model with current weights?
See original GitHub issueHi,
While I am running a pretrained CRDNN model, I finetune it on some data with .fit() method. Then, I would like to see its performance with .evaluate() function.
However, when I run the evaluate() function the log is something like this:
0%| | 0/16 [00:00<?, ?it/s]
6%|β | 1/16 [00:22<05:42, 22.84s/it]
12%|ββ | 2/16 [00:29<03:03, 13.12s/it]
19%|ββ | 3/16 [00:54<04:04, 18.84s/it]
25%|βββ | 4/16 [01:21<04:23, 21.98s/it]
31%|ββββ | 5/16 [01:47<04:18, 23.45s/it]
38%|ββββ | 6/16 [02:17<04:17, 25.71s/it]
44%|βββββ | 7/16 [02:43<03:50, 25.65s/it]
50%|βββββ | 8/16 [03:11<03:31, 26.41s/it]
56%|ββββββ | 9/16 [03:20<02:27, 21.08s/it]
62%|βββββββ | 10/16 [03:26<01:37, 16.31s/it]
69%|βββββββ | 11/16 [03:40<01:17, 15.56s/it]
75%|ββββββββ | 12/16 [04:08<01:18, 19.58s/it]
81%|βββββββββ | 13/16 [04:38<01:08, 22.67s/it]
88%|βββββββββ | 14/16 [04:50<00:38, 19.27s/it]
94%|ββββββββββ| 15/16 [04:57<00:15, 15.83s/it]
100%|ββββββββββ| 16/16 [05:35<00:00, 22.38s/it]
100%|ββββββββββ| 16/16 [05:35<00:00, 20.98s/it]
speechbrain.utils.train_logger - Epoch loaded: 15 - test loss: 1.50, test CER: 31.43, test WER: 37.69
As you can see, it seems that model is evaluated on Epoch 15 which probably indicates that it loads weights from that epoch instead of using the weights which have just been finetuned. Whenever I do this finetuning and then evaluate(), the scores are exactly the same (up to 8 decimal digits). How do I overcome that?
TLDR; I want to use current weights with .evaluate() function, not with weights loaded from best epoch. How do I achieve that?
Issue Analytics
- State:
- Created a year ago
- Comments:7
Top Results From Across the Web
Training and evaluation with the built-in methods - TensorFlow
This guide covers training, evaluation, and prediction (inference) models when using built-in APIs for training & validation (such asΒ ...
Read more >Evaluate the Performance of Deep Learning Models in Keras
In this post, you will discover a few ways to evaluate model performance using Keras. Kick-start your project with my new book Deep...
Read more >Step 4: Build, Train, and Evaluate Your Model
Training involves making a prediction based on the current state of the model, calculating how incorrect the prediction is, and updating theΒ ...
Read more >keras: load saved model weights in a model for evaluation
After training, I load the model weights in to a model for evaluation, but I found the model does not give the best...
Read more >How to use a model to do predictions with Keras - ActiveState
Specifies what layers the model contains, and how they are connected. Weights. Input parameters that influence output in a Keras model.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
(In practice, you can, I believe, just pass an argument that forces to use the latest instead of the best using the keys).
brain.checkpointer=None
works. I looked over the source code forbrain.checkpointer.delete_checkpoints(num_to_keep=0)
but this delets the checkpoints from the directory which is not exactly what I want to do.