pipenv won't install airflow

Issue description

When attempting to install airflow, pipenv refuses to create a lock file, even though (I think) there’s not actually a problem

> mkdir airflow-pipenv
> cd airflow-pipenv
> pipenv install airflow[mysql]==1.8.0
Creating a virtualenv for this project...
Pipfile: /Users/mgrenderjones/Documents/workspace/airflow-pipenv/Pipfile
... snip snip...
Adding airflow[mysql]==1.8.0 to Pipfile's [packages]...
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches jinja2<2.9.0,==2.8.1,>=2.10,>=2.4,>=2.7.3,>=2.8
Tried: 2.0, 2.1, 2.1.1, 2.2, 2.2.1, 2.3, 2.3.1, 2.4, 2.4.1, 2.5, 2.5.1, 2.5.2, 2.5.3, 2.5.4, 2.5.5, 2.6, 2.7, 2.7.1, 2.7.2, 2.7.3, 2.8, 2.8, 2.8.1, 2.8.1, 2.9, 2.9, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 2.9.3, 2.9.3, 2.9.4, 2.9.4, 2.9.5, 2.9.5, 2.9.6, 2.9.6, 2.10, 2.10

But it’s not clear to me that there’s a >=2.10.0 dependency:

> pipenv graph | grep -i jinja
    - Jinja2 [required: >=2.4, installed: 2.8.1]
      - Jinja2 [required: >=2.4, installed: 2.8.1]
      - Jinja2 [required: >=2.4, installed: 2.8.1]
      - Jinja2 [required: >=2.4, installed: 2.8.1]
      - Jinja2 [required: >=2.4, installed: 2.8.1]
      - Jinja2 [required: >=2.4, installed: 2.8.1]
  - jinja2 [required: >=2.7.3,<2.9.0, installed: 2.8.1]
    - Jinja2 [required: >=2.8, installed: 2.8.1]

Expected result

Airflow should install (with Jinja2==2.8.1)

Actual result

Seems I can’t have airflow and a lock file.

Looking at the output of pipenv lock --verbose (and setup.py) the issue appears to be that airflow requires a restricted set of versions of Jinja2 (<=2.9.0) as well as specific versions of flask and it’s helpers e.g. flask-[admin|swagger] (there are more), which themselves require >=2.10 of Jinja2. And so kaboom.

But my project doesn’t use flask-admin etc, and I’ve been happily using airflow for a long time now without issue - is there no way to force the issue? Sure, we could say this is an ‘upstream’ problem, but it feels like there ought to be a way to say ‘please let me do this’!

Steps to replicate

See above

---

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/Users/mgrenderjones/Library/Python/2.7/lib/python/site-packages/pipenv'

Python location: '/usr/local/opt/python@2/bin/python2.7'

Other Python installations in PATH:

• 2.7: /Users/mgrenderjones/.local/share/virtualenvs/vulcan-pipenv-vMW1Zc4p/bin/python2.7

• 2.7: /Users/mgrenderjones/.local/share/virtualenvs/vulcan-pipenv-vMW1Zc4p/bin/python2.7

• 2.7: /usr/local/bin/python2.7

• 2.7: /usr/bin/python2.7

• 2.7.15: /Users/mgrenderjones/.local/share/virtualenvs/vulcan-pipenv-vMW1Zc4p/bin/python

• 2.7.15: /usr/local/bin/python

• 2.7.10: /usr/bin/python

• 2.7.15: /Users/mgrenderjones/.local/share/virtualenvs/vulcan-pipenv-vMW1Zc4p/bin/python2

• 2.7.15: /usr/local/bin/python2

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64',
 'python_full_version': '2.7.15',
 'python_version': '2.7',
 'sys_platform': 'darwin'}

System environment variables:

• _OLD_VIRTUAL_PATH
• SPARK_HOME
• PYTHONDONTWRITEBYTECODE
• LOGNAME
• USER
• HOME
• PATH
• TERM_PROGRAM
• LANG
• TERM
• Apple_PubSub_Socket_Render
• SHLVL
• XPC_FLAGS
• _OLD_FISH_PROMPT_OVERRIDE
• PIPENV_ACTIVE
• TERM_SESSION_ID
• XPC_SERVICE_NAME
• SSH_AUTH_SOCK
• VIRTUAL_ENV
• SHELL
• TERM_PROGRAM_VERSION
• PIP_PYTHON_PATH
• TMPDIR
• __CF_USER_TEXT_ENCODING
• PWD

Pipenv–specific environment variables:

• PIPENV_ACTIVE: 1

Debug–specific environment variables:

• PATH: /Users/mgrenderjones/.local/share/virtualenvs/vulcan-pipenv-vMW1Zc4p/bin:/Users/mgrenderjones/.cargo/bin:/Users/mgrenderjones/Library/Python/2.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/munki
• SHELL: /usr/bin/fish
• LANG: en_US.UTF-8
• PWD: /Users/mgrenderjones/Documents/workspace/airflow-pipenv
• VIRTUAL_ENV: /Users/mgrenderjones/.local/share/virtualenvs/vulcan-pipenv-vMW1Zc4p

---

Contents of Pipfile (‘/Users/mgrenderjones/Documents/workspace/airflow-pipenv/Pipfile’):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]

[packages]
airflow = {version = "==1.8.0", extras = ["mysql"]}

[requires]
python_version = "2.7"

[arocketman]

hi everyone, this is kind of unrelated but since the google search for “Pipenv airflow fails” brings to here I think I will also write that I solved my (dumb) problem by making sure I was doing

pipenv install apache-airflow

and not…

pipenv install airflow

😃

[uranusjr]

I’m closing this to guide people to #2596. That thread contains much more information, and should be much more useful to everyone.