remote execution with local python package

Hi,

demo project clearml demo server runs

I have a project with local package involving source compilation that need to be install via pip. The requirements.txt specifies the local package as: ./local_package

I’d like to run clearml-agent on such project but the package detection stage seems to only report the name of the local package and not the location as it would do for packages installed from a git repository.

So when I try to execute such a task with clearml-agent, the installation fails:

ERROR: Could not find a version that satisfies the requirement my_local_package==0.0.0 (from -r /tmp/cached-reqsj92bjyak.txt (line 4)) (from versions: none)
ERROR: No matching distribution found for my_local_package==0.0.0 (from -r /tmp/cached-reqsj92bjyak.txt (line 4))

clearml_agent: ERROR: Could not install task requirements!

I can still edit on the webui the install packages to add ./local_package but this is cumbersome. Moreover, after exectution, the install packages field stores an absolute path for this local package which can break replication on another worker.

A solution would be to move local_package on a git repository and install it through pip install git+... but my team is not willing to do it for their own reasons.

What do you think ?

[bmartinn]

Just updating here that an RC is out 😃

pip install clearml==-0.17.6rc1

[bmartinn]

This task is now likely to fail (or run with a wrong version) if we try to run it on another worker as the path for my_local_package is probably not valid.

You have a very good point, this is definitely a bug. I’m updating here once the clearml-agent fixes this issue (basically updating back should replace back the local package link)