[Feature request] Support click CLI

I am using ‘click’ instead of argparse for my CLI, therefore command line arguments are not recognised as such. This causes problems when using clearml-agent and remote execution. If i understood correctly, the task is pulled by the agent and after environment setup, the worker calls the training script again, with the command line arguments catched from ArgumentParser. Using click, this leads to the problem, that command line arguments are not recorded, and the remote worker fails, because no command line arguments are provided. My current workaround is, to use default arguments and task.connect() on those, such that the default arguments are updated after the task has been started by the agent. However, this solution does not allow to use ‘required’ command line arguments.

The following MWE, when called with ‘python mwe.py --dataset CUSTOM_DATASET’ would print ‘CUSTOM_DATASET’ on my local machine but on the remote machine it would use the default parameter ‘DEFAULT_DATASET’. Making the parameter --dataset required, the script would crash.

import clearml
import click

@click.command()
@click.option('--dataset', type=str, default='DEFAULT_DATASET')
def main(dataset):

    print(dataset)

    task = clearml.Task.init(project_name='example_project', task_name='example_task')
    task.execute_remotely(queue_name='example_queue')


if __name__ == '__main__':
    main()

[klekass]

Works! 🥳 Thanks!

[bmartinn]

Hi @klekass A fix was pushed, it should work now 🤞 , could you verify on your code ?

pip install git+https://github.com/allegroai/clearml.git