Optimize conan install --update

Running conan install -u on my conanfile that links in 10 different packages takes almost 30 seconds when nothing gets updated. My guess is that it does something like:

for p in package:
  hash(p)
  check_local(p)
  check_remote(p)

I think this can probably be optimized so that if it’s the hashes that take time: do them in parallel, if it’s the calls to remote we probably should extend the API to do fewer calls to check more info.

Why is this needed?

For a release build I wipe my conan dir on the CI to make sure I have the latest and greatest even if the version number hasn’t changed. But for local developers I also want to make sure that it’s fast to check for updates on all binary packages so that they don’t sit and spend a lot of time debugging something already fixed.

Ideally our cmake run could execute conan install -u everytime - that requires the execution to go down from 30 seconds to around 1-2 seconds though.

Let me know if I can do some local profiling.

[memsharded]

Thanks very much @cubanpit for testing and following up. Indeed seems a case on the Gitlab side. You can link the issue here if you report, so we can track it too. I’d possibly recommend keeping your ArtifactoryCE around, at least locally, specially because Conan 2.0 works exclusively with API v2, and Gitlab hasn’t implemented it yet, so it will not be possible to test it (even in Conan 1.X with revisions enabled, to be ready for 2.0). And 2.0 is looking fantastic, it will be a great improvement.

I will leave this ticket open, because the original issue of potential optimization of --update is still valid.

[cubanpit]

Here are the traces in the two cases: conan_trace_artifactory.log conan_trace_gitlab.log

This seems to confirm a bug in the GitLab implementation unfortunately. Thank you a lot for your support on the way. I wonder if there is any way to work-around this behavior, in the meantime I will submit a bug report to GitLab.

EDIT: Link to GitLab issue: https://gitlab.com/gitlab-org/gitlab/-/issues/366425