Can't access old parameters including python type

I have a use-case where I need to access the parameters of a previous experiment, including the original python type:

Example: I use Argparse to define my model (layers, activation functions, optimizers parameters, etc) and do things like choosing how often I log data based on iteration number. Maybe after training for n-epochs, I see the model hasn’t converged, so I want to train it some more.

If I want to reload this experiment, I can use: previous_task = Task.get_task(task_id=<TASK-ID>)

and continue an experiment with: task = Task.init('examples', 'continue-run example', continue_last_task=TASK-ID>)

But I also need the parameters I would have gotten from Argparse to rebuild the model correctly! If I use: previous_task.get_parameter() I see the original Argparse parameters I used, but the value are all strings. i.e. I can just plug in these values when I build the model.

What is the correct way to continue experiments with the old parameters? i.e.

• Move the previous_task parameters to an Argparse’ args.dict?
• Not use Argparse directly, but somehow directly access the task parameters after its processed the Argpass inputs?

Is this related to issue #260 ?

[dnhkng]

Sure, go ahead and rename the issue!

[bmartinn]

I see a way to do it using the GUI now. When I clone an experiment I keep my old parameters (clearml-agent magic!). I could use the GUI to edit the “args.continue_model_id” parameter from “None” to the original experiment ID, and continue the training just by adding this code to my standard training script this code:

Yes! That’s exactly what I had in mind, very cool!

A second use case for getting parameters back is the practical use of the model!

Yep that would actually work quite well, though I have to point to the fact the actual types are defined on the Argparse object definition, and the knowledge both Args and currently instantiated Argparse use the same arguments, is implicit (which is good for job security 😉 but probably something we should avoid). My only idea for improving this use case is actually storing the MyModelClass instance on the “training task” so that the “inference task” could just use the exact same object. Now obviously pickling the class assumes the class code is available, but this will at least guarantee that we are using the same object. You can achieve that by just storing the MyModelClass with torch.save with a different filename (automagic will create a new model with a different name), or use the task.upload_artifact which will auto-pickle the class-object and upload it as an artifact, whatever makes more sense to you.

wdyt?

BTW: can we rename the issue to “adding type casting to get_parameters_as_dict()” (or something similar)?