Pipenv doesn't install dependencies of top-level dependencies from pre-existing Pipfile

Here is the story: I’m migrating from old requirements.txt to a brand new Pipfile. First time I run pipenv install, it said me that there is requirements.txt and it will convert it. OK, I thought. Next, it installed dependencies.

Then I tried to play with my project, run some code, and it fails due to missing imports. “It is strange”, I thought. Then I run pipenv graph and find that some of dependencies of direct dependencies of my project was not installed.

I did a bunch of experiments combining different options, pythons (2.7, 3.6),Pipfile’s (and .lock’s) and still can’t figure out why pipenv doesn’t install all the dependencies.

Then I do stupid thing: I grabbed list of top-level deps, removed Pipfile{,.lock} and requirements.txt and installed all of them step-by-step with pipenv install <package>. Funny, but it worked.

Describe your environment

1. Solus
2. Python version: Python 2.7.14
3. Pipenv version: pipenv, version 10.1.0

Expected result

I expect that when I install deps from pre-existing Pipfile, pipenv install all of them, including deps of top-level deps.

Actual result

Some of deps of top-level deps are not installed at all.

Steps to replicate

Pipfile:

[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]

cryptography = "*"
APScheduler = "==3.0.5"
"pathlib2" = "*"


[dev-packages]



[requires]

python_version = "2.7"

Then:

$ pipenv install --two
$ pipenv graph | egrep 'installed: \?'

Trying to import:

$ python -m pathlib2
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "<venv>/lib/python2.7/site-packages/pathlib2.py", line 45, in <module>
    from scandir import scandir as os_scandir
ImportError: No module named scandir

[tribals]

Also, it seems like even if I manually re-installed package to fix dependencies problem, such originally missing deps does not locked in Pipfile.lock. Which means that the main goal of pipenv - reproducible, deterministic reestablishment of application environment - not achieved at all.

$ pipenv install
$ pipenv graph | egrep 'installed: \?'
  - futures [required: Any, installed: ?]
  - enum34 [required: Any, installed: ?]
  - ipaddress [required: Any, installed: ?]
  - scandir [required: Any, installed: ?]
$ pipenv uninstall apscheduler; pipenv install apscheduler
$ pipenv graph | egrep 'installed: \?' # seems like now dependency is installed
  - enum34 [required: Any, installed: ?]
  - ipaddress [required: Any, installed: ?]
  - scandir [required: Any, installed: ?]
$ grep futures Pipfile.lock || echo ALARM! Dependency is not locked!
ALARM! Dependency is not locked!

[abulka]

I updated to the latest version 11.5.2 (not sure why brew gave me an older version yesterday).

I’ve also become aware that people can get tricked if there is an existing Pipfile.lock with missing dependencies - then of course you are going to get missing dependencies when you pipenv install because pipenv install only looks at the lockfile.

So I have been careful to pipenv update which should fix up (add any needed) dependencies in the lockfile, yet I am still getting missing dependencies from the lockfile, and thus from the pipenv graph. I totally reinstalled pipenv using brew, pipenv --rm to remove the pipenv virtual env and rebooted, treid again, but am still getting the missing dependencies. So I ran update with the -v verbose option and saw:

...
...
  found candidate stripe==1.79.1 (constraint was <any>)
  found candidate treelib==1.5.1 (constraint was <any>)
  found candidate whitenoise==2.0.6 (constraint was ==2.0.6)

Finding secondary dependencies:
  sphinx-js==2.3.1          requires -
  sphinx==1.7.1             requires -
  requests-mock==1.4.0      requires -
  furl==1.0.1               requires -
  jsonpickle==0.9.6         requires -
  pytz==2018.3              requires -
  psycopg2==2.6.1           requires -
  dj-database-url==0.4.0    requires -
  attrs==17.4.0             requires -
  maya==0.3.4               requires -
  mock==2.0.0               requires -
  recommonmark==0.4.0       requires -
  sphinx-rtd-theme==0.1.9   requires -
  django-allauth==0.35.0    requires -
  treelib==1.5.1            requires -
  beautifulsoup4==4.6.0     requires -
  pyyaml==3.12              requires -
  django==1.11.8            requires -
  gunicorn==19.4.5          requires -
  django-extensions==2.0.5  requires -
  selenium==3.10.0          requires -
  django-sendgrid-v5==0.6.87 requires -
  whitenoise==2.0.6         requires -
  plantweb==1.1.0           requires -
  requests==2.18.4          requires -
  stripe==1.79.1            requires -
  astpretty==1.2.1          requires -
  sphinxcontrib-plantuml==0.6 requires -
------------------------------------------------------------
Result of round 1: stable, done
Warning: Error generating hash for mock
Warning: Error generating hash for recommonmark
Warning: Error generating hash for sphinx-rtd-theme
Warning: Error generating hash for sphinx-js
...
...

The secondary dependencies are coming up blank? On my other mac machine, the dependencies detect what’s needed and go through various ‘rounds’ and it all works ok. The machine which has problems is a solid Mac machine with python 2 and python 3 installed using brew. No sure how to debug this or whether this deserves another open issue (apologies if I’ve hijacked the OP, but it seems related).