Are there asynchronous parallel examples?

I have a function that needs to be evaluate remotely

evaluate(parameter, server_id):
    # send `parameter` to server based on `server_id`
    return {'metric': (res, 0)}

For instance, I have 5 servers and want to optimize the metric in parallel. Currently the only way I could think of is to run the following :

import multiprocessing as mp
ps=[]
for i in range(5):
    parameters, trial_index = ax_client.get_next_trial()
    p = mp.Process(target=evaluate, args=(parameters, i))
    p.start()
    ps.append(p)
for i in range(5):
    ps[i].join()
    # retrieve data 
    ax_client.complete_trial(trial_index=trial_index, raw_data=retrieved_data)

This is actually synchronous, which is less efficient. Is there a way to construct an asynchronous parallel evaluation?

[mpolson64]

Hi @deng-cy thank you for reaching out! Your use case may be a good candidate for our Scheduler API, which has a tutorial here.

It involves a little more overhead to set up compared to the Service API, you will define a Runner object to send trials to your external system and Metric objects to retrieve the data once the trial has been run, but it gives you the ability to set maximum parallelism (among other settings) and allow Ax to handle scheduling, new point generation, and trial deployment and polling for you automatically. Personally this is my favorite way of using Ax as it allows the user to “set it and forget it”, which I find more than worth the effort in setting the Scheduler up for most experiments.

If you need any assistance setting this up or any other questions with your specific use case please feel free to continue to respond in this thread.

[deng-cy]

@sgbaird Yeah Service API, I corrected it.