Pipfile needs a way to manually specify package installation order

GDAL is a package that is used by many GIS projects. During installation, GDAL’s python bindings are compiled using the headers from the existing numpy version. So you need to install numpy before GDAL, else numpy bindings will not work.

This seems to be a use case for pipenv install --sequential, but I am not able to control the order of the sequential install.

NB: you need to have libgdal-dev on Ubuntu 16 and CPLUS_INCLUDE_PATH=/usr/include/gdal and C_INCLUDE_PATH=/usr/include/gdal to reproduce this.

pipenv install numpy gdal==1.11.2 --sequential works as expected.

Then, pipenv --rm, pipenv install --deploy --sequential gives me the wrong sequence: numpy is installed after gdal, and my numpy bindings do not work.

The sequence seems to origin from the Pipfile.lock, which is alphabetic. gdal is before numpy in the alphabet, so at least installation fails in a reproducible way.

How should I approach this issue?

Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
numpy = "*"
gdal = "==1.11.2"

[dev-packages]

[requires]
python_version = "2.7"

Pipfile.lock:

{
    "_meta": {
        "hash": {
            "sha256": "42b5387796abc5d9542f6213df851d564959e44cd1c1a0789fc9dc5e0e5f5d0f"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "2.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "gdal": {
            "hashes": [
                "sha256:7827197836690815b0142412a49a97c6c6a18b21a83ece8cd6d8ec66127f10df"
            ],
            "index": "pypi",
            "version": "==1.11.2"
        },
        "numpy": {
            "hashes": [
                "sha256:0074d42e2cc333800bd09996223d40ec52e3b1ec0a5cab05dacc09b662c4c1ae",
                "sha256:034717bfef517858abc79324820a702dc6cd063effb9baab86533e8a78670689",
                "sha256:0db6301324d0568089663ef2701ad90ebac0e975742c97460e89366692bd0563",
                "sha256:1864d005b2eb7598063e35c320787d87730d864f40d6410f768fe4ea20672016",
                "sha256:46ce8323ca9384814c7645298b8b627b7d04ce97d6948ef02da357b2389d6972",
                "sha256:510863d606c932b41d2209e4de6157ab3fdf52001d3e4ad351103176d33c4b8b",
                "sha256:560e23a12e7599be8e8b67621396c5bc687fd54b48b890adbc71bc5a67333f86",
                "sha256:57dc6c22d59054542600fce6fae2d1189b9c50bafc1aab32e55f7efcc84a6c46",
                "sha256:760550fdf9d8ec7da9c4402a4afe6e25c0f184ae132011676298a6b636660b45",
                "sha256:8670067685051b49d1f2f66e396488064299fefca199c7c80b6ba0c639fedc98",
                "sha256:9016692c7d390f9d378fc88b7a799dc9caa7eb938163dda5276d3f3d6f75debf",
                "sha256:98ff275f1b5907490d26b30b6ff111ecf2de0254f0ab08833d8fe61aa2068a00",
                "sha256:9ccf4d5c9139b1e985db915039baa0610a7e4a45090580065f8d8cb801b7422f",
                "sha256:a8dbab311d4259de5eeaa5b4e83f5f8545e4808f9144e84c0f424a6ee55a7b98",
                "sha256:aaef1bea636b6e552bbc5dae0ada87d4f6046359daaa97a05a013b0169620f27",
                "sha256:b8987e30d9a0eb6635df9705a75cf8c4a2835590244baecf210163343bc65176",
                "sha256:c3fe23df6fe0898e788581753da453f877350058c5982e85a8972feeecb15309",
                "sha256:c5eb7254cfc4bd7a4330ad7e1f65b98343836865338c57b0e25c661e41d5cfd9",
                "sha256:c80fcf9b38c7f4df666150069b04abbd2fe42ae640703a6e1f128cda83b552b7",
                "sha256:e33baf50f2f6b7153ddb973601a11df852697fba4c08b34a5e0f39f66f8120e1",
                "sha256:e8578a62a8eaf552b95d62f630bb5dd071243ba1302bbff3e55ac48588508736",
                "sha256:f22b3206f1c561dd9110b93d144c6aaa4a9a354e3b07ad36030df3ea92c5bb5b",
                "sha256:f39afab5769b3aaa786634b94b4a23ef3c150bdda044e8a32a3fc16ddafe803b"
            ],
            "index": "pypi",
            "version": "==1.14.3"
        }
    },
    "develop": {}
}

[simonk52]

(I’ve just stumbled across this same problem, thanks for the pointer to pygdal)

The way I see it, not all projects can be relied on to write their setup.py scripts in the “right” way. Therefore, it would be really nice to be able to override the dependencies of any package in the Pipfile. Somehow we would need to specify that, for the current application, gdal depends on numpy. Would something like this be possible?

[packages]
gdal = { setup_requires = ["numpy"] }

[mattskone]

This is not a proper solution, but it might be acceptable for some until (if) this feature is implemented.

pipenv install --dev appears to install dev-packages before the default packages, so after editing the Pipfile by placing the numpy dependency in dev-packages, and GDAL in packages, then pipenv install --dev --sequential seems to install numpy before GDAL and avoids the author’s original error.