Upgrade ADS requirements to be compatible with ivadomed

In order to run axondeepseg and ivadomed in the same environment, we would need to upgrade the following requirements:

pillow==6.2.0 to at least ==7.0.0 scikit-learn==0.19.2 to at least ==0.20.3 tqdm== 4.24.0 to at least ==4.30 numpy==1.16.4 to at least ==1.16.6

ADS tests won’t pass with pillow==7.0.0.

Details:

test/test_integrity_test.py:17: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
AxonDeepSeg/integrity_test.py:40: in integrity_test
    prediction = axon_segmentation([path_testing], ["image.png"], path_model, config_network, prediction_proba_activate=True, verbosity_level=4)
AxonDeepSeg/apply_model.py:258: in axon_segmentation
    gpu_per=gpu_per, verbosity_level=verbosity_level)
AxonDeepSeg/apply_model.py:51: in apply_convnet
    path_acquisitions, acquisitions_resolutions, resampled_resolutions, verbose_mode=verbosity_level)
AxonDeepSeg/apply_model.py:326: in load_acquisitions
    original_acquisitions.append(ads.imread(path_img))
AxonDeepSeg/ads_utils.py:261: in imread
    raw_img = imageio.imread(filename)
../miniconda/envs/ads_ivado_a/lib/python3.7/site-packages/imageio/core/functions.py:206: in imread
    reader = read(uri, format, 'i', **kwargs)
../miniconda/envs/ads_ivado_a/lib/python3.7/site-packages/imageio/core/functions.py:123: in get_reader
    format = formats.search_read_format(request)
../miniconda/envs/ads_ivado_a/lib/python3.7/site-packages/imageio/core/format.py:673: in search_read_format
    if format.can_read(request):
../miniconda/envs/ads_ivado_a/lib/python3.7/site-packages/imageio/core/format.py:188: in can_read
    return self._can_read(request)
../miniconda/envs/ads_ivado_a/lib/python3.7/site-packages/imageio/plugins/pillow.py:97: in _can_read
    Image = self._init_pillow()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Format PNG-PIL - Portable network graphics>

    def _init_pillow(self):
        with self._lock:
            if not self._pillow_imported:
                self._pillow_imported = True  # more like tried to import
                import PIL
                if not hasattr(PIL, 'PILLOW_VERSION'):  # pragma: no cover
>                   raise ImportError('Imageio Pillow requires '
                                      'Pillow, not PIL!')
E                   ImportError: Imageio Pillow requires Pillow, not PIL!

../miniconda/envs/ads_ivado_a/lib/python3.7/site-packages/imageio/plugins/pillow.py:81: ImportError

But all tests passed with the latest version of pillow==7.2.0 and the versions mentioned above for the three other libraries.

[joshuacwnewton]

Short answer

to not have to maintain the versions in both ivadomed and ADS, i suggest we remove the version requirement on ADS for packages that are already present in ivadomed

I very much agree with loosening install_requires within setup.py for ADS. But, I’m not sure I agree with the suggestion to stop maintaining requirements entirely for ADS. I think it’s worthwhile to still try to specifiy the minimal requirements, just like IVADOMED does.

and install ivadomed before these packages (so that the versions required by ivadomed will be specified).

If both ADS and IVADOMED have compatible minimal requirements (which I think is likely – the range of acceptable versions can be quite large, see IVADOMED), then in theory they should be able to be installed together without needing to specify an order.

Longer answer

Both projects are currently parsing requirements.txt directly for install_requires inside setup.py.

https://github.com/neuropoly/axondeepseg/blob/b0c06509a6dd612837830c36a0e04f0f3aff22ae/setup.py#L17-L20

https://github.com/neuropoly/axondeepseg/blob/b0c06509a6dd612837830c36a0e04f0f3aff22ae/setup.py#L50

But, requirements.txt and setup.py are meant to have very different purposes. Here’s a very comprehensive StackOverflow answer that explains why, and here’s its summary:

• setup.py should declare the loosest possible dependency versions that are still workable. Its job is to say what a particular package can work with.
• requirements.txt is a deployment manifest that defines an entire installation job, and shouldn’t be thought of as tied to any one package. Its job is to declare an exhaustive list of all the necessary packages to make a deployment work.
• Because these two things have such different content and reasons for existing, it’s not feasible to simply copy one into the other.

I’m guessing that ADS used pip freeze > requirements.txt at some point. But, by bringing these strict requirements over to setup.py, I believe conflicts were practically inevitable (hence #327).

So, my longer-term suggestion would be to use @mariehbourget’s findings for minimum version requirements directly in setup.py, and stop using requirements.txt for the pip install.

[jcohenadad]

thank you so much for the very descriptive answer @joshuacwnewton ! 🙏