Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Hyperparameter optimization does not return learning rate

See original GitHub issue

Presently, the hyperparameter optimization returns the best parameters for

{'gradient_clip_val': 1.028043566346387,
 'hidden_size': 176,
 'dropout': 0.22095396475678628,
 'hidden_continuous_size': 36,
 'attention_head_size': 1}

even when use_learning_rate_finder=True. I expect this could be fixed easily just by adding it to the optuna parameters to track.

Issue Analytics

State:
Created 3 years ago
Comments:10 (3 by maintainers)

Top GitHub Comments

2reactions

jdb78commented, Oct 20, 2020

Annoying mishap. Pushed a fix.

1reaction

jdb78commented, Oct 21, 2020

Yes. It’s unreleased.

Top Results From Across the Web

How to Configure the Learning Rate When Training Deep ...

In this tutorial, you will discover the learning rate hyperparameter used when training deep learning neural networks.

18. Hyperparameter Tuning

Learning rate is a hyperparameter that controls the step size to move in the direction of lower loss function, with the goal of...

Automated Machine Learning Hyperparameter Tuning in Python

Increasingly, hyperparameter tuning is done by automated methods that aim to find optimal hyperparameters in less time using an informed search ...

Optuna Guide: How to Monitor Hyper-Parameter Optimization ...

Time-consuming – in deep learning, where one experiment takes hours to complete, this strategy is not efficient. Another traditional approach is ...

Optimizer Benchmarking Needs to Account for ... - arXiv

Budget for hyperparameter optimization (# models trained) ... and classification, tuning only the learning rate for Adam is the most reliable option, ...