[KED-1055] Dependency issue with pyarrow

Description

There is a regression on windows for kedro 0.15.1 and 0.15.0 on windows. When running pipenv install kedro===0.14.3, everything is fine, but when doing pipenv install kedro===0.15.0 or pipenv install kedro===0.15.1, it fails at a depency on pyarrow 0.14.1

Context

I can’t install the latest version of kedro on windows 10

Steps to Reproduce

1. create a fresh python environment and activate it
2. pipenv install kedro

Expected Result

The dependency should be resolved, and kedro 0.15 should be able to install on windows

Actual Result

Fails to install kedro

[pipenv.exceptions.InstallError]: ['Collecting pyarrow==0.14.1 (from -r C:\\Users\\OUELAL\\AppData\\Local\\Temp\\pipenv-1e42tyv4-requirements\\pipenv-ocm118ym-requirement.txt (line 1))']
[pipenv.exceptions.InstallError]: ['ERROR: Could not find a version that satisfies the requirement pyarrow==0.14.1 (from -r C:\\Users\\OUELAL\\AppData\\Local\\Temp\\pipenv-1e42tyv4-requirements\\pipenv-ocm118ym-requirement.txt (line 1)) (from versions: 0.4.1, 0.5.0, 0.6.0, 0.7.0, 0.7.1, 0.8.0, 0.9.0, 0.10.0, 0.11.0, 0.11.1, 0.12.0, 0.12.1, 0.13.0, 0.14.0)', 'ERROR: No matching distribution found for pyarrow==0.14.1 (from -r C:\\Users\\OUELAL\\AppData\\Local\\Temp\\pipenv-1e42tyv4-requirements\\pipenv-ocm118ym-requirement.txt (line 1))']

Your Environment

Include as many relevant details about the environment in which you experienced the bug:

• Kedro version used (pip show kedro or kedro -V):
• Python version used (python -V): Python 3.6.8
• Operating system and version: Windows 10 Pro

[AlexandreOuellet]

I’ve had a look, and pyarrow was pinned to 0.12.0 in the kedro 0.14.3 release.

Also, this doesn’t happen with a pip install, only pipenv. I’ve had a similar issue with airflow, where essentially an unpinned dependency got updated, and it’s requirements of a common library was pinned to a higher version. So airflow depended on X and Y. X also depended on an unpinned version of Y and suddenly the dependency got all screwed up when X got updated. Pip was not making an error (although it should have), but pipenv was correctly throwing an error.

edit: To clarify, it was essentially this issue : https://github.com/pypa/pipenv/issues/2596

[AlexandreOuellet]

as a workaround, you can pin the pyarrow version to 0.14.0, and it seems to do the trick.

Here is my Pipfile :

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pyarrow = "==0.14.0"
kedro = "*"

[requires]
python_version = "3.6"

What I find strange is that you need to pin the pyarrow version within the packages for it to work.