setup.py not compatible for setting up a development installation on a windows machine

As per setup guide , pip install -e . fails to install optuna for development on a windows machine/environment.

Expected behavior

It should install optuna with dev dependencies without any errors.

Environment

• Windows 10/11
• python 3.9.1
• pip 21.2.4
• setuptools 58.2.0
• wheel 0.37.0
• optuna : current master

Error messages, stack traces, or logs

Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: 'c:\python\python.exe' 'c:\python\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py' get_requires_for_build_wheel 'C:\Users\91728\AppData\Local\Temp\tmprb137rje'
      cwd: C:\Users\91728\Desktop\optuna\optuna
Complete output (22 lines):
  Traceback (most recent call last):
    File "c:\python\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 349, in <module>
      main()
    File "c:\python\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 331, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "c:\python\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 117, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\91728\AppData\Local\Temp\pip-build-env-h54rpc43\overlay\Lib\site-packages\setuptools\build_meta.py", line 154, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "C:\Users\91728\AppData\Local\Temp\pip-build-env-h54rpc43\overlay\Lib\site-packages\setuptools\build_meta.py", line 135, in _get_build_requires
      self.run_setup()
    File "C:\Users\91728\AppData\Local\Temp\pip-build-env-h54rpc43\overlay\Lib\site-packages\setuptools\build_meta.py", line 258, in run_setup
      super(_BuildMetaLegacyBackend,
    File "C:\Users\91728\AppData\Local\Temp\pip-build-env-h54rpc43\overlay\Lib\site-packages\setuptools\build_meta.py", line 150, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 194, in <module>
      long_description=get_long_description(),
    File "setup.py", line 25, in get_long_description
      return f.read()
    File "c:\python\lib\encodings\cp1252.py", line 23, in decode
      return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 2146: character maps to <undefined>

Steps to reproduce

1. Use windows machine/environment
2. git clone ########/optuna.git
3. cd optuna
4. pip install -e .

Additional context

The UnicodeDecodeError happens here because generally text encoding in a windows machine is cp1252. You can see output of [System.Text.Encoding]::Default from my machine below.

IsSingleByte      : True
BodyName          : iso-8859-1
EncodingName      : Western European (Windows)
HeaderName        : Windows-1252
WebName           : Windows-1252
WindowsCodePage   : 1252
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : True
CodePage          : 1252

And, encoding of README.md file is UTF-8.

$ file --mime-encoding README.md
README.md: utf-8

So, I think get_long_description function fails to parse the file in a windows environment. My insights are derived from here.

So, changing this line to with open(readme_filepath, encoding="utf8") as f: would hopefully resolve this error.

I would love to open a PR with the changes if this feels suitable.

[toshihikoyanase]

Thank you for reporting the issue. I was not aware of that since I usually develop optuna on WSL2. UnicodeDecodeError occurred because the encoding default was cp932.

As a workaround, I successfully installed optuna from the source with PYTHONUTF8=1.

> $env:PYTHONUTF8 = 1
> pip install -e .

Not a strong opinion, but it may be too early to modify the code for Windows. The reasons are as follows:

1. Optuna repository doesn’t have CI environment for Windows as @nzw0301 mentioned. Most of the committers use macOS or Linux, and we may not be able to review changes for Windows.
2. The issue affects only developers on Windows. Users can simply install optuna with pip install optuna.
3. We have the workaround to avoid UnicodeDecodeError.

[nzw0301]

I suppose @himkt @toshihikoyanase use Windows. What do you think about this proposal 🙇🏻 ?