Default encoding for .env files

I had some Russian symbols in my .env file while launching pipenv shell from Windows 10 and got the following result:

$ pipenv shell
Loading .env environment variables…
Traceback (most recent call last):
  File "c:\program files (x86)\python36-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\program files (x86)\python36-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Program Files (x86)\Python36-32\Scripts\pipenv.exe\__main__.py", line 9, in <module>
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\click\core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\cli.py", line 564, in shell
    core.load_dot_env()
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\core.py", line 155, in load_dot_env
    dotenv.load_dotenv(denv, override=True)
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\dotenv\main.py", line 250, in load_dotenv
    return DotEnv(f, verbose=verbose).set_as_environment_variables(override=override)
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\dotenv\main.py", line 94, in set_as_environment_variables
    for k, v in self.dict().items():
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\dotenv\main.py", line 73, in dict
    values = OrderedDict(self.parse())
  File "c:\program files (x86)\python36-32\lib\site-packages\pipenv\vendor\dotenv\main.py", line 80, in parse
    for line in f:
  File "c:\program files (x86)\python36-32\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 426: character maps to <undefined>

If it helps, here’s the help output

$ python -m pipenv.help

$ python -m pipenv.help output

Pipenv version: '11.10.0'

Pipenv location: 'C:\\Program Files (x86)\\Python36-32\\lib\\site-packages\\pipenv'

Python location: 'C:\\Program Files (x86)\\Python36-32\\python.exe'

Other Python installations in PATH:

• 3.6: C:\Program Files (x86)\Python36-32\python.exe

• 3.6.4: C:\Program Files (x86)\Python36-32\python.exe

• 3.6.4: C:\Windows\py.exe

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.4',
 'os_name': 'nt',
 'platform_machine': 'AMD64',
 'platform_python_implementation': 'CPython',
 'platform_release': '10',
 'platform_system': 'Windows',
 'platform_version': '10.0.16299',
 'python_full_version': '3.6.4',
 'python_version': '3.6',
 'sys_platform': 'win32'}

System environment variables:

• ACLOCAL_PATH
• ALLUSERSPROFILE
• APPDATA
• COMMONPROGRAMFILES
• COMPUTERNAME
• COMSPEC
• CONFIG_SITE
• COMMONPROGRAMFILES(X86)
• COMMONPROGRAMW6432
• DISPLAY
• EXEPATH
• FPS_BROWSER_APP_PROFILE_STRING
• FPS_BROWSER_USER_PROFILE_STRING
• FSHARPINSTALLDIR
• HOME
• HOMEDRIVE
• HOMEPATH
• HOSTNAME
• INFOPATH
• LANG
• LEIN_JAVA_CMD
• LOCALAPPDATA
• LOGONSERVER
• MANPATH
• MINGW_CHOST
• MINGW_PACKAGE_PREFIX
• MINGW_PREFIX
• MSYSTEM
• MSYSTEM_CARCH
• MSYSTEM_CHOST
• MSYSTEM_PREFIX
• NUMBER_OF_PROCESSORS
• ORIGINAL_PATH
• ORIGINAL_TEMP
• ORIGINAL_TMP
• OS
• ONEDRIVE
• PATH
• PATHEXT
• PKG_CONFIG_PATH
• PLINK_PROTOCOL
• PROCESSOR_ARCHITECTURE
• PROCESSOR_ARCHITEW6432
• PROCESSOR_IDENTIFIER
• PROCESSOR_LEVEL
• PROCESSOR_REVISION
• PROGRAMFILES
• PS1
• PSMODULEPATH
• PUBLIC
• PWD
• PROGRAMDATA
• PROGRAMFILES(X86)
• PROGRAMW6432
• SESSIONNAME
• SHELL
• SHLVL
• SSH_ASKPASS
• SYSTEMDRIVE
• SYSTEMROOT
• TEMP
• TERM
• TMP
• TMPDIR
• USERDOMAIN
• USERDOMAIN_ROAMINGPROFILE
• USERNAME
• USERPROFILE
• WINDIR
• _
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH
• PYTHONUNBUFFERED

Pipenv▒specific environment variables:

Debug▒specific environment variables:

• PATH: C:\Users\feana\bin;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\local\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\feana\bin;C:\Program Files\Docker\Docker\Resources\bin;C:\ProgramData\Oracle\Java\javapath;C:\Program Files (x86)\Python36-32\Scripts;C:\Program Files (x86)\Python36-32;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\Program Files\dotnet;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files\Git\cmd;C:\Users\feana\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Microsoft VS Code\bin;C:\Users\feana\.lein\bin;C:\Program Files\Git\usr\bin\vendor_perl;C:\Program Files\Git\usr\bin\core_perl
• SHELL: C:\Program Files\Git\usr\bin\bash.exe
• LANG: en_US.UTF-8
• PWD: C:/Users/feana

---

Expected result

Correct .env file parsing.

Actual result

Somehow, it thinks the file is encoded in CP1252. Why?..

Steps to replicate

1. Be on Win10
2. Have cyrillic in .env
3. Run pipenv shell

[uranusjr]

@feakuru Yes, this should not happen. Post the traceback and I’ll work on it. Regarding the env file, what encoding do you expect it to be decoded with? UTF-8? (And for your question, it does not know what the file is encoded in, so it simply uses the system default, i.e. the code page encoding on Windows.)

[uranusjr]

@techalchemy Urrgh, I never got to work on this in the end… This is still not fixed, sorry.