Single Quotes in Pipfile are not handled correctly in latest versions

Issue description

The latest two releases (2021.11.9 and 2021.11.15) no longer support single quotes around environment variables (in order to handle special characters in the environment variables) in Pipfiles (example here of what did work in previous versions: https://pipenv-fork.readthedocs.io/en/latest/advanced.html#injecting-credentials-into-pipfiles-via-environment-variables)

Based on the error messages, the single quotes get converted into %27 in the strings.

[GPHemsley-RELX]

@matteius I’m not sure I understand what you’re saying. The issue here is how to deal with a password that contains special characters that, when the environment variable is expanded, could interfere with parsing the URL.

Previously, the practice was to put single quotes around it, with any necessary magic being taken care of by pipenv. Then it was changed to require the special characters to be pre-escaped in the environment variable.

So, for example, if your password was foo@bar, in v2021.5.29 you could do this:

PYPI_PASSWORD=foo@bar

[[source]]
url = "https://matteius:'${PYPI_PASSWORD}'@example.com/pypi"

But in v2021.11.5 you have to manually do the url-escaping yourself:

PYPI_PASSWORD=foo%40bar

[[source]]
url = "https://matteius:${PYPI_PASSWORD}@example.com/pypi"

Which means you can’t use PYPI_PASSWORD anywhere that expects a literal (un-url-escaped) password.

I have not confirmed whether anything has changed since then, as I believe we instead changed our password to not contain special characters.

[GPHemsley-RELX]

For reference, it looks like this was previously raised in #4856 and then the documentation was adjusted in 1524315baf9610d1b4732fff265625c47ef6375f to say that the password must be manually percent-encoded.

However, if your password is coming from an environment variable, it is entirely possible that it is also used outside of this usecase, where the percent encoding may not be wanted/allowed. Additionally, the quotation advice was previously listed in the documentation as best practice, which means dropping support for it is a BC break.

At the very least, this change should be listed more prominently, especially because the error message it causes does not in any way indicate where the problem is coming from.

But beyond that, I feel that a solution which percent encodes automatically would be better. (I presume that is what was being done before?)

To be clear: this worked in v2021.5.29 and stopped working in v2021.11.5.

The error message manifests as:

[pipenv.exceptions.InstallError]: WARNING: 401 Error, Credentials not correct for ...