need easy way to move a run in repo to another

🚀 Feature

If there’s two repo with different directory, it will be helpful if there’s easy way to move one’s repo to another.

Motivation

This request is concerned with question below: https://app.slack.com/client/T015QFY3ZNZ/C01DK8FM2P3/thread/C01DK8FM2P3-1642569350.057200

Currently, if you want to change experiment’s repo, you need to: 1. move experiment data to another repo.

    # $hash is hashed valued of the experiment
    mv /aim/.aim/seqs/progress/$hash /another_aim_repo/.aim/seqs/progress/
    mv /aim/.aim/seqs/locks/$hash another_aim_repo/.aim/seqs/locks/ 
    mv /aim/.aim/seqs/chunks/$hash another_aim_repo/.aim/seqs/chunks/ 
    mv /aim/.aim/meta/chunks/$hash another_aim_repo/.aim/meta/chunks/ 
    mv /aim/.aim/meta/locks/$hash another_aim_repo/.aim/meta/locks/ 

2. add index through the code below

    from aim import Run, Repo
    repo = Repo.from_path("another_aim_repo")
    run = Run(run_hash=$hash, repo=repo, system_tracking_interval=None)
    # however, there also problem if you don't add experiment in above line, all migrated experiment will set their name to 'default'
    # run = Run(run_hash=$hash, repo=repo, system_tracking_interval=None,experiment='migrated_experiemnt')

Pitch

A = repo.from_path("a") # have runs named "run_a,run_b,run_c"
B = repo.from_path("b") # have runs named "run_d,run_e,run_f"


A.migrate(B, experiment="run_d")
# now repo A have runs "run_a,run_b,run_c,run_d"

A.migrate(B)
# now repo A  have runs "run_a,run_b,run_c,run_d,run_e,run_f"

Alternatives

Additional context

This issue were driven by thoughts to merge all experiments from multiple repos to one It would be much easy if all experiment were executed in one repo. I tried to host aim through k8s, in a way guided here: https://aimstack.readthedocs.io/en/latest/guides/integrations/basic_k8s_deployment.html However, nfs I used did not support writing on sqlite, so this configuration didn’t work for me. It would not be needed if I could change database but I found that all system were hard-coded to use sqlite. Maybe support to use various database system can be another solution

[mihran113]

Hey @ptaejoon. The feature you’ve requested is shipped with aim v.3.5.0. There are now options to copy/move runs from one repo to another with either CLI or SDK.

1. Using CLI: aim runs --repo=<source_repo_path> cp --destination=<destination_repo_path> run_hash_1, run_hash_2 ... aim runs --repo=<source_repo_path> mv --destination=<destination_repo_path> run_hash_1, run_hash_2 ... Source repo path defaults to current working directory. Destination repo path and at least one run hash are required.

2. Using SDK:

from aim import repo
A = repo.from_path("a")
B = repo.from_path("b")

A.copy_runs([run_hash_1, run_hash2], B)
# A.move_runs([run_hash_1, run_hash2], B)

[vakker]

This is giving me an error. If I run from the .aim parent dir aim runs cp bfb90fe96d284a07b5334e3e --destination ../aim then it gives me:

Error while trying to copy run 'bfb90fe96d284a07b5334e3e'. 'No key () is present.'.
Something went wrong while copying runs. Remaining runs are:
bfb90fe96d284a07b5334e3e

If I use *, then it just goes through all of the hashes:

$ aim runs cp '*' --destination ../aim
Error while trying to copy run 'bfb90fe96d284a07b5334e3e'. 'No key () is present.'.
Error while trying to copy run 'e6d285dfc4f146fb8561fb54'. 'No key () is present.'.
Error while trying to copy run 'dc5358d0254a41cd8bf79041'. 'No key () is present.'.
Error while trying to copy run '34338c65e82a42e3bd454147'. 'No key () is present.'.
Error while trying to copy run 'cc0b79624eb34c1e90d3e781'. 'No key () is present.'.
Error while trying to copy run 'f0f567701e99455e93250e19'. 'No key () is present.'.
Error while trying to copy run '98b7c31867aa4546b3dcb1c1'. 'No key () is present.'.
Error while trying to copy run 'f55f31cbfca147ecb4553101'. 'No key () is present.'.
Error while trying to copy run '7f3855e44e2f4831a726a9bb'. 'No key () is present.'.
Error while trying to copy run 'b9b6338ef9d343be9314233c'. 'No key () is present.'.
...