pipenv install with private source doesn't use provided environment variables

Expected result

Have configured a private source which require username and password to authenticate. (It’s a privately hosted https://jfrog.com/artifactory/ pypi repo.)

[[source]]
url = "http://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@my-private-repo"
verify_ssl = false
name = "artifactory"

When running ARTIFACTORY_USERNAME=myuser ARTIFACTORY_PASSWORD=mypass pipenv install from the same directory as the pipfile on a project, I would expect the env-variables in the private source to use the values I provided.

I’ve also tried to provide env-variables in other way:

• Setting env variable inline on the command (as shown in the example command above)
• Exporting env variables
• Putting env variables into a ‘.env’ file that look like this:

ARTIFACTORY_USERNAME=myuser
ARTIFACTORY_PASSWORD=mypass

All of the above give me the same result error result (below).

When I provide the username and password directly inline in the pipfile it works fine, but when I use environment variables instead it fails with the following error:

Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple --extra-index-url http://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@my-private-repo --trusted-host my-private-repo

                          ROUND 1                           
Current constraints:
  aiopubsub==1.0.0
  asynctest
  pytest==3.3.2

Finding the best candidates:
  found candidate aiopubsub==1.0.0 (constraint was ==1.0.0)
User for artifactory.firmglobal.com: 
INFO:pip9._vendor.requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): artifactory.firmglobal.com
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pipenv/resolver.py", line 82, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/pipenv/resolver.py", line 71, in main
    clear=do_clear,
  File "/usr/local/lib/python2.7/dist-packages/pipenv/resolver.py", line 63, in resolve
    verbose=verbose,
  File "/usr/local/lib/python2.7/dist-packages/pipenv/utils.py", line 425, in resolve_deps
    pre,
  File "/usr/local/lib/python2.7/dist-packages/pipenv/utils.py", line 336, in actually_resolve_reps
    resolved_tree.update(resolver.resolve(max_rounds=PIPENV_MAX_ROUNDS))
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/resolver.py", line 102, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/resolver.py", line 193, in _resolve_one_round
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/resolver.py", line 193, in <setcomp>
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/resolver.py", line 257, in get_best_match
    best_match = self.repository.find_best_match(ireq, prereleases=self.prereleases)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/repositories/pypi.py", line 116, in find_best_match
    all_candidates = self.find_all_candidates(ireq.name)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/piptools/repositories/pypi.py", line 101, in find_all_candidates
    candidates = self.finder.find_all_candidates(req_name)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/index.py", line 456, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/index.py", line 606, in _get_pages
    page = self._get_page(location)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/index.py", line 722, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/index.py", line 831, in get_page
    "Cache-Control": "max-age=600",
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/pip9/_vendor/requests/sessions.py", line 488, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/pip9/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/pip9/_vendor/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/pip9/_vendor/requests/sessions.py", line 602, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/pip9/_vendor/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/pip9/download.py", line 181, in handle_401
    username = six.moves.input("User for %s: " % parsed.netloc)
EOFError: EOF when reading a line

/usr/local/lib/python2.7/dist-packages/pipenv/utils.py:1230: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/pipenv-ip4PTc-requirements'>
  warnings.warn(warn_message, ResourceWarning)

I assume it’s just a confusing error message to tell me that the authentication fails, since I get the exact same error message if I put in an invalid username or password directly in the pipfile private source description.

Steps to replicate

pipfile which use source that require authentication. Use env variables for credentials:

[[source]]
url = "http://${ARTIFACTORY_USERNAME}:${ARTIFACTORY_PASSWORD}@private-repo"
verify_ssl = false
name = "my repo"

Run pipenv install where you provide credentials: ARTIFACTORY_USERNAME=user ARTIFACTORY_PASSWORD=pass pipenv install

What am I doing wrong?

$ python -m pipenv.help output

Pipenv version: '11.9.0'

Pipenv location: '/usr/local/lib/python2.7/dist-packages/pipenv'

Python location: '/usr/bin/python'

Other Python installations in PATH:

• 2.7: /usr/bin/python2.7

• 2.7: /usr/bin/python2.7

• 3.5: /usr/bin/python3.5m

• 3.5: /usr/bin/python3.5

• 3.6: /usr/bin/python3.6m

• 3.6: /usr/bin/python3.6

• 2.7.12: /usr/bin/python

• 2.7.12: /usr/bin/python2

• 3.5.2: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.4.0-116-generic',
 'platform_system': 'Linux',
 'platform_version': '#140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018',
 'python_full_version': '2.7.12',
 'python_version': '2.7',
 'sys_platform': 'linux2'}

System environment variables:

• LC_NUMERIC
• MANDATORY_PATH
• CONSCRIPT_OPTS
• _LXSESSION_PID
• XDG_GREETER_DATA_DIR
• SPARK_HOME
• PROJECT_HOME
• LC_CTYPE
• PYTHONDONTWRITEBYTECODE
• XDG_CURRENT_DESKTOP
• LC_PAPER
• LOGNAME
• XDG_SEAT
• PATH
• XDG_VTNR
• QT_PLATFORM_PLUGIN
• PYTHONUNBUFFERED
• VIRTUALENVWRAPPER_SCRIPT
• ZSH
• DISPLAY
• XDG_SESSION_DESKTOP
• LANG
• CONFIRMIT_CFG_DIR
• TERM
• SHELL
• XDG_SESSION_PATH
• XAUTHORITY
• LANGUAGE
• SHLVL
• QT_QPA_PLATFORMTHEME
• TERMINATOR_UUID
• PIP_REQUIRE_VIRTUALENV
• WINDOWID
• _
• SSH_AGENT_PID
• HOME
• LC_MONETARY
• XDG_CONFIG_HOME
• SAL_USE_VCLPLUGIN
• CONSCRIPT_HOME
• XDG_RUNTIME_DIR
• WORKON_HOME
• VIRTUALENVWRAPPER_PROJECT_FILENAME
• LC_ADDRESS
• SSH_AUTH_SOCK
• VIRTUALENV_PYTHON
• GDMSESSION
• VIRTUALENVWRAPPER_WORKON_CD
• CONFIRMIT_CFG_HOME
• IBUS_DISABLE_SNOOPER
• XDG_SEAT_PATH
• PIP_PYTHON_PATH
• XDG_SESSION_ID
• DBUS_SESSION_BUS_ADDRESS
• ORBIT_SOCKETDIR
• VIRTUALENVWRAPPER_HOOK_DIR
• LC_IDENTIFICATION
• DESKTOP_SESSION
• LSCOLORS
• XDG_CONFIG_DIRS
• DEFAULTS_PATH
• XDG_SESSION_TYPE
• OLDPWD
• LS_COLORS
• GDM_LANG
• LC_TELEPHONE
• LC_MEASUREMENT
• PWD
• COLORTERM
• LC_NAME
• XDG_MENU_PREFIX
• LC_TIME
• LESS
• PAGER
• XDG_DATA_DIRS
• USER

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /home/andreas/.conscript/bin:/opt/spark-2.2.1/bin:/opt/flink/bin:/home/andreas/activator-dist-1.3.12/bin:/home/andreas/bin:/home/andreas/bin:/home/andreas/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
• SHELL: /usr/bin/zsh
• LANG: en_US.UTF-8
• PWD: /path-to-project

---

Contents of Pipfile (‘/path-to-project/Pipfile’):

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "http://$USERNAME:$PASSWORD@my-private-repo"
verify_ssl = false
name = "artifactory"

[dev-packages]
pytest = "==3.3.2"
asynctest = "*"
aiopubsub = "==1.0.0"

[packages]
aio-pika = "==1.8.1"
deepdiff = "==3.3.0"
pika = "*"
async-timeout = "*"
async-generator = "*"
"e1839a8" = {path = ".", editable = true}

[requires]
python_version = "3.6"

[Gr1N]

Found an issue https://github.com/pypa/pipenv/issues/2037

[Gr1N]

@techalchemy version 11.10.1.dev1 works fine for me thanks! Also as I can see you need to updated docs:

Luckily - pipenv will hash your Pipfile before expanding environment variables (and, helpfully, will substitute the environment variables again when you install from the lock file - so no need to commit any secrets! Woo!)

It’s not true anymore, as I see in Pipfile.lock env vars not hashed and it’s great to my mind:

        ...
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            },
            {
                "name": "artifactory",
                "url": "${EXTRA_PYPI_INDEX}",
                "verify_ssl": true
            }
        ]
        ...