Unknown metric error when subclassing tuner
See original GitHub issueI am trying to subclass a tuner, e.g. Hyperband, to tune batch_size using the following
class MyTuner(kt.tuners.Hyperband):
def run_trial(self, trial, *args, **kwargs):
kwargs['batch_size'] = trial.hyperparameters.Int('batch_size', 1024, 4096, step=512)
super(MyTuner, self).run_trial(trial, *args, **kwargs)
With this, tuner.search
gives the following error at the end of the first trial:
Epoch 1/2
59/59 [==============================] - 2s 32ms/step - loss: 5.1755 - accuracy: 0.7180 - val_loss: 0.5713 - val_accuracy: 0.9060
Epoch 2/2
59/59 [==============================] - 2s 29ms/step - loss: 0.4279 - accuracy: 0.9235 - val_loss: 0.2897 - val_accuracy: 0.9425
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: `Tuner.run_trial()` returned None. It should return one of float, dict, keras.callbacks.History, or a list of one of these types. The use case of calling `Tuner.oracle.update_trial()` in `Tuner.run_trial()` is deprecated, and will be removed in the future.
if sys.path[0] == '':
---------------------------------------------------------------------------
/usr/local/lib/python3.7/dist-packages/keras_tuner/engine/metrics_tracking.py in _assert_exists(self, name)
284 def _assert_exists(self, name):
285 if name not in self.metrics:
--> 286 raise ValueError("Unknown metric: %s" % (name,))
287
288
ValueError: Unknown metric: val_accuracy
The same exact code works if I use Hyperband without subclassing. This happens independently of the model type (tried with a CNN and an LSTM) or the metric used (e.g. tried with accuracy
, loss
, val_loss
).
To Reproduce
https://colab.research.google.com/drive/1-H9PFZwObxpV5Jw04JhlqLHfuz3ncgyJ?usp=sharing
Expected behaviour
Tuning should continue and batch size should be tuned.
Issue Analytics
- State:
- Created 2 years ago
- Comments:7
Top Results From Across the Web
Unknown metric val_accuracy using Keras Tuner Error
I am trying to use a Custom Keras Tuner with fashion-mnist dataset in Google Colab, that's my code: !pip install keras-tuner import ...
Read more >Getting started with KerasTuner
Here, we use mean squared error (MSE) as an example. First, we implement the MSE metric by subclassing keras.metrics.Metric .
Read more >Unknown metric val_accuracy using Keras Tuner
Anything I'm doing wrong? I am getting val_accuracy values in training process, but when the first trial finishes it just stops with this...
Read more >Training (tune.Trainable, session.report) — Ray 2.2.0
With the Function API, you can report intermediate metrics by simply calling ... The Trainable class API will require users to subclass ray.tune.Trainable...
Read more >Automatic Hyperparameter Optimization With Keras Tuner
Learn how to utilize the search algorithms of Keras Tuner to ... change the model in response to the estimated error each time...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Found the problem, please change your code to
Just to mention that adding the return statement worked for me.