Special characters not supported in the authors section of pyproject.toml

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10 17763.253

• Poetry version: 0.12.11

• pyproject.toml file: https://gist.github.com/taljaards/b06c1dda1ece0b8d443eb9512213e47d

Issue

Poetry does not support special characters in the “authors” section of the pyproject.toml file. I created a new folder, did poetry init and answered no to most of the yes/no questions.

Afterwards, when doing poetry add, it fails:

C:\Users\Stephan\Source\jupyter-lab>poetry add jupyterlab -vvv

[UnicodeDecodeError]
'utf-8' codec can't decode byte 0xe9 in position 88: invalid continuation byte

Exception trace:
 C:\Users\Stephan\.poetry\lib\poetry\_vendor\py3.7\cleo\application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 C:\Users\Stephan\.poetry\lib\poetry\console\application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 C:\Users\Stephan\.poetry\lib\poetry\_vendor\py3.7\cleo\application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 C:\Users\Stephan\.poetry\lib\poetry\console\commands\command.py in run() at line 77
   return super(BaseCommand, self).run(i, o)
 C:\Users\Stephan\.poetry\lib\poetry\_vendor\py3.7\cleo\commands\base_command.py in run() at line 136
   self.initialize(input_, output_)
 C:\Users\Stephan\.poetry\lib\poetry\console\commands\env_command.py in initialize() at line 18
   current_env = Env.get(self.poetry.file.parent)
 C:\Users\Stephan\.poetry\lib\poetry\console\commands\command.py in poetry() at line 62
   return self.get_application().poetry
 C:\Users\Stephan\.poetry\lib\poetry\console\application.py in poetry() at line 60
   self._poetry = Poetry.create(os.getcwd())
 C:\Users\Stephan\.poetry\lib\poetry\poetry.py in create() at line 98
   local_config = TomlFile(poetry_file.as_posix()).read()
 C:\Users\Stephan\.poetry\lib\poetry\_vendor\py3.7\tomlkit\toml_file.py in read() at line 20
   return loads(f.read())
 C:\Users\Stephan\AppData\Local\Programs\Python\Python37\lib\codecs.py in decode() at line 322
   (result, consumed) = self._buffer_decode(data, self.errors, final)

add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] [--] <name> (<name>)...

This is because of the “é” character in my name. After removing the “é”, operations using the pyproject.toml file work as expected.

[nuno-andre]

Another é-named here.

I think the problem (also for #221) is that subprocess calls git pipelining through Windows stdio, which hasn’t native support for UTF-8, the MinGW default encoding. Hence the replacement character.

From Python 3.7 (PEP 540) this can be circumvented by setting the environment variable PYTHONUTF8 which makes Python to ignore local encoding using UTF-8.

So, this annoyance for us the beyondasciians (sorry) can be avoided in cmd with:

set PYTHONUTF8=1 && poetry init

Or for posh:

$env:PYTHONUTF8=1; poetry init

[taljaards]

@vlcinsky Yup, it works! Thanks.