Can't create Pipfile.lock for cu111 torch distribution

Issue description

Hello !

Pipenv 2022.08.24 can’t create the lock file for the torch package distributed on https://download.pytorch.org/whl/cu111

I replicated the issue with this minimal Dockerfile, but it also happens on my host machine:

FROM nvcr.io/nvidia/tensorrt:22.07-py3

ENV LANG C.UTF-8
ENV PIPENV_NOSPIN 1

RUN pip install -qU --no-cache-dir pip pipenv && \
    pipenv --support && \
    pipenv install -vvv --index https://download.pytorch.org/whl/cu111 torch

ENTRYPOINT ["/bin/sh", "-c"]
CMD pipenv shell

Expected result

Should build an image where torch is installed and Pipfile and Pipfile.lock are created

Actual result

See pipenv-bug.log

Steps to replicate

In the same directory as the Dockerfile provided above:

docker build . --network=host --progress=plain 2> pipenv-bug.log

---

$ pipenv --support

Pipenv version: '2022.8.24'

Pipenv location: '/usr/local/lib/python3.8/dist-packages/pipenv'

Python location: '/usr/bin/python'

OS Name: 'posix'

User pip version: '22.2.2'

user Python installations found:

• 3.8.10: /usr/bin/python3
• 3.8.10: /usr/bin/python
• 3.8.10: /usr/bin/python3.8
• 3.8.10: /bin/python3
• 3.8.10: /bin/python
• 3.8.10: /bin/python3.8

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.10',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.15.57.1-microsoft-standard-WSL2',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Wed Jul 27 02:20:31 UTC 2022',
 'python_full_version': '3.8.10',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

• NPP_VERSION
• SHELL
• NVIDIA_VISIBLE_DEVICES
• DALI_BUILD
• CUSOLVER_VERSION
• CUBLAS_VERSION
• CUFFT_VERSION
• NVIDIA_REQUIRE_CUDA
• CUDA_CACHE_DISABLE
• NCCL_VERSION
• CUSPARSE_VERSION
• PIPENV_NOSPIN
• ENV
• PWD
• TENSORRT_VERSION
• OPENUCX_VERSION
• NSIGHT_SYSTEMS_VERSION
• NVIDIA_DRIVER_CAPABILITIES
• TRT_VERSION
• NVIDIA_PRODUCT_NAME
• RDMACORE_VERSION
• HOME
• LANG
• CUDA_VERSION
• CURAND_VERSION
• CUTENSOR_VERSION
• HPCX_VERSION
• DLPROF_VERSION
• GDRCOPY_VERSION
• OPENMPI_VERSION
• NVJPEG_VERSION
• LIBRARY_PATH
• SHLVL
• BASH_ENV
• CUDNN_VERSION
• NSIGHT_COMPUTE_VERSION
• DALI_VERSION
• LD_LIBRARY_PATH
• NVIDIA_BUILD_ID
• OMPI_MCA_coll_hcoll_enable
• OPAL_PREFIX
• CUDA_DRIVER_VERSION
• _CUDA_COMPAT_PATH
• NVIDIA_TENSORRT_VERSION
• TENSORRT_OSS_VERSION
• PATH
• MOFED_VERSION
• TRTOSS_VERSION
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PIP_PYTHON_PATH
• PYTHONDONTWRITEBYTECODE
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

• PIPENV_NOSPIN: 1

Debug–specific environment variables:

• PATH: /opt/tensorrt/bin:/usr/local/mpi/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin
• SHELL: /bin/bash
• LANG: C.UTF-8
• PWD: /workspace

[matteius]

My docker build of your example also failed with the same error you got. The windows lock finished on 3.8 and got real disk heavy during the end of the lock, and I could see some heavy memory usage in powershell at that time, exceeding 2GB at times.

PS C:\Users\matte\Projects\pipenv-triage\pipenv-5293> cat .\Pipfile.lock
{
    "_meta": {
        "hash": {
            "sha256": "8c8a8ab26523f27f69e3c0974ba0b0415a21e7fd4ab84e60eb83fa90b7d2b125"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            },
            {
                "name": "downloadpytorch",
                "url": "https://download.pytorch.org/whl/cu111",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "torch": {
            "hashes": [
                "sha256:4b4bf40d844fdcf9b7261e62aff9eede7c89c2b904343b3f9dc12393763779cb",
                "sha256:566cbfad47b25ef6b1023518982179439d8e53069f3d2341e8adcd1e4b2ef179",
                "sha256:588188a37e2f2c9bb8f807e0d49ac4d66b9ef23f4417233247c5d2f796c4f42d",
                "sha256:78ae2260ac1902c5a224fe1b7994072bbfa7ef8a1734d8697ca74249fa6391c5",
                "sha256:7c23faff945e98c6e242ce41a480e7fe48426140d5994ea3b9886c02145d58cf",
                "sha256:bbe445308a43960efeabc4f4a4b6c5d0db209ef5d5148672c750943d1ea42f54",
                "sha256:bcb611272d7aa5c2f7c1648b2227303a023b1de46aabccf1846a8aca1bfb5cd8",
                "sha256:ccbd72f39d76fb7e7db6513bcc8e4ee684348f9865bd8cddca6ce2dd07b3ccd0"
            ],
            "index": "downloadpytorch",
            "version": "==1.10.2+cu111"
        },
        "typing-extensions": {
            "hashes": [
                "sha256:25642c956049920a5aa49edcdd6ab1e06d7e5d467fc00e0506c44ac86fbfca02",
                "sha256:e6d2677a32f47fc7eb2795db1dd15c1f34eff616bcaf2cfb5e997f854fa1c4a6"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==4.3.0"
        }
    },
    "develop": {}
}

Did you not have this problem on prior versions of pipenv?

[adrizein]

Hi, thank you for the quick response. Here is what I have for df -h:

Filesystem      Size  Used Avail Use% Mounted on
none            3.9G  4.0K  3.9G   1% /mnt/wsl
/dev/sdd       1007G   17G  939G   2% /mnt/wsl/docker-desktop-data/isocache
drivers         931G  370G  561G  40% /usr/lib/wsl/drivers
none            3.9G     0  3.9G   0% /usr/lib/wsl/lib
none            3.9G   12K  3.9G   1% /mnt/wsl/docker-desktop/shared-sockets/host-services
/dev/sdc       1007G   61M  956G   1% /mnt/wsl/docker-desktop/docker-desktop-user-distro
/dev/sde       1007G   56G  900G   6% /
rootfs          3.9G  1.9M  3.9G   1% /init
none            3.9G     0  3.9G   0% /dev
none            3.9G  4.0K  3.9G   1% /run
none            3.9G     0  3.9G   0% /run/lock
none            3.9G     0  3.9G   0% /run/shm
none            3.9G     0  3.9G   0% /run/user
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
drvfs           931G  370G  561G  40% /mnt/c
/dev/loop0      359M  359M     0 100% /mnt/wsl/docker-desktop/cli-tools

I don’t think it is a disk space issue, as I am able to install torch/torchvision and many more packages with a pip only approach.