question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to learn which model Ax used for a given optimization?

See original GitHub issue

Hi, I am using the below code to optimize my neural network model and I am able to see the optimized results. I can also use the best parameters given by AE and improve my model. However, for the purpose of explanation in the research paper, I am not able to figure out which type of optimization was actually used in the multi-arm bandit configuration of AE client.

ax_client = AxClient()
# create the experiment.
ax_client.create_experiment( name="keras_experiment", parameters=parameters, objective_name='keras_cv', minimize=True)
no_of_trials=12
for i in range(no_of_trials):
    parameters, trial_index = ax_client.get_next_trial()
    ax_client.complete_trial(trial_index=trial_index, raw_data=evaluate(parameters,train_X,train_y,val_X, val_y))
# save results to json file.
ax_client.save_to_json_file()

ax_client.get_trials_data_frame().sort_values('trial_index') # PRINTING/VISUALIZING THE RESULTS - look at all the trials.
#VIEW THE BEST VALUES OF HYPERPARAMETERS
best_parameters, values = ax_client.get_best_parameters()
# the best set of parameters.
for k in best_parameters.items():
  print(k)
print()
# the best score achieved.
means, covariances = values
print(means)
#plot the evolution of the keras_cv score over iterations. view the score reaches the minimum value 
render(ax_client.get_optimization_trace()) 

Please help / provide the resource which can help me understand which algorithm was actually run ? Was it Bayesian Opt or some other algorithm?

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
bhaskatripathicommented, Nov 2, 2021

Hi @lena-kashtelyan , I am going through the link you shared. Will get back to you in case I have any questions.

0reactions
lena-kashtelyancommented, Nov 4, 2021

@bhaskatripathi, please reopen this issue if you have any follow-ups : ) BO_MIXED doesn’t use EB or TS, so just wanted to make sure nobody walked away from this issue confused about that!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Design Optimization with Ax in Python - Towards Data Science
This code uses Bayesian Optimization to iteratively explore a state space and fit a Gaussian Process to the underlying model (experiment).
Read more >
Hyperparameter-tuning for Deep Learning Models with the Ax
A quick comparison of the results is given at the end of this article to see the effectiveness of using the Bayesian Optimization...
Read more >
Bayesian Optimization · Ax - Adaptive Experimentation Platform
Bayesian optimization starts by building a smooth surrogate model of the outcomes using Gaussian processes (GPs) based on the (possibly noisy) observations ...
Read more >
Simple Settings to Help Your AX 2009 or AX 2012 ... - YouTube
Long-term use and stability of Dynamics AX depends on a healthy system. Join this webcast to learn the settings and user features that ......
Read more >
Six Steps to Optimize Dynamics AX Performance - YouTube
Time is money. Nowhere is this adage more true than in the competitive world of manufacturing where productivity and system inefficiencies ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found