Getting your run dependencies right

Hey folks,

Something I said in the comments of the merged RIOS PR:

I’m not convinced you should relax your gdal 2.0.2 spec (or perhaps it should become a >= spec). The TravisCI output suggests that rios isn’t working with gdal 2.0.0. You need to prevent people from installing this package alongside 2.0.0, then.

I wanted to make sure that this doesn’t get lost in an already closed/merged thread, because it doesn’t just apply to RIOS, it applies across the entire conda-forge ecosystem.

It is important that a package’s run requirements have sufficient version requirements to ensure proper operation. In particular: it is not sufficient to assume that a user will have, or obtain, the latest version of a given dependency. If you know your package is not compatible with older versions of a dependency, you must reflect that in the run dependencies.

For example: suppose I have a functioning environment built containing gdal 2.0.0 from defaults; then I type conda install rios. If rios simply specifies gdal as a run-time dependency, with no version specification, then it will happily install it without upgrading gdal. But rios will be broken according to this. This means that an unqualified runtime dependency on gdal is incorrect. On the other hand, if rios includes gdal >2.0 as its runtime dependency, it will cause conda to upgrade past the failure.

On the other hand, you want your dependencies to be as relaxed as possible, otherwise you invite a litany of unsatisfiable specs in complex environments. So, for example, the current rios recipe specifies gdal 2.0.2. In my view, this ought to be relaxed to gdal >2.0 or gdal >=2.0.2, unless you have good reason to believe that subsequent versions of gdal will break rios.

To be fair, the problems here are made worse by channel collision issues: it could be that conda-forge’s version of gdal 2.0* works just fine, but the ones in default do not. We’re working on that, as you know. But for the foreseeable future, people are going to be mixing and matching packages from defaults and conda-forge, so one should still take care to lock down dependencies to avoid breakage.

[pelson]

I think this issue has served its purpose of raising awareness of the issue. In the interest of keeping things clean, I’m going to go ahead and close this. I think there is a place for these kinds of lessons in documentation, be it on conda somewhere, or in the fledgling docs at conda-forge.

Thanks for raising. 👍

[msarahan]

Overly specific pinning has been the cause of many problems I’ve dealt with in supporting anaconda. Please avoid pinning to a specific version. Automation might make it easier, but I still agree with @mcg1969 that this is more trouble down the road. Please be patient, and let’s do this right. I’ll push this internally as best I can, and having @mcg1969 on board should help. I will invite him to the meeting for discussing this.