[tune] Cannot create an implementation based on HyperOpt or Optuna with Ray Tune

What is your question?

I’m trying to create an implementation of Raytune with Hyperopt, I have gone through the provided tutorial, however this is not relevant for how I require my hyperopt implementation. I’m optimizing a Deep Neural Net architecture and use the HyperOpt fmin function for this (this is different than from the example provided for RayTune iirc). When I do try to base me of this example from the documentation then I keep on getting an Attribute error: “metric unknown” when I try to maximize the accuracy.

Example of how I run my search space in HyperOpt: fn = create_model creates the DNN model of which the architecture is to be optimized. Space holds all the possible hyperparameters that are to be considered. best_run = fmin(fn = create_model, space = test_space, algo = tpe.suggest, max_evals = 10, trials = trials)

Regarding the Optuna library there is no example provided, although this post claims that there are 4 different search algorithm available for Optuna. Can you point me in the right direction for this?

Ray version and other system information (Python version, TensorFlow version, OS): Python Version: 3.6 Tensorflow: 1.14 Ray: 0.8.6 OS: macOS Mojave 10.14.6

[richardliaw]

RE: hparams: Hmm, maybe try pip install -U tensorboardX?

RE: accuracy: please make the following change -

validation_acc = np.amax(result.history['val_acc'])
  
- tune.report(**{'loss': -validation_acc, 'status': STATUS_OK, 'model': model})
+ tune.report(**{'accuracy': validation_acc, 'status': STATUS_OK, 'model': model})

That should fix things for you!

[richardliaw]

Hey there, this issue is quite stale. Can we open a new issue to track? thanks!

On Sun, Aug 28, 2022 at 7:40 AM knilakshan20 @.***> wrote:

@richardliaw https://github.com/richardliaw I got a similar issue when reporting a dictionary as follows. `tune.report(**{‘harrells_c’:np.amax(results.history[‘harrells_c’])})

TensorFlow 2.8.2 ray 2.0.0

`model = builder(D,config)

model.compile(loss=losses[‘task1’], metrics=[metrics[‘task1’]], optimizer=optimizer) model.summary()

results = model.fit(x=dataset.batch(256), epochs=epochs, verbose=verbose, ) `

— Reply to this email directly, view it on GitHub https://github.com/ray-project/ray/issues/10142#issuecomment-1229475062, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCRZZKY4YGIPIQJFCTMG4TV3N26RANCNFSM4QA3QMCA . You are receiving this because you were mentioned.Message ID: @.***>