setuptools (39.1.0) doesn't set up Description-Content-Type correctly on Windows

Originally reported in pypa/warehouse#4341.

It looks like long_description is full of Windows-style carriage-return characters (^M) which seem to have confused twine/pkginfo about what is and what isn’t part of long_description.

My project at PyPI is at: https://pypi.org/project/gff3tool/1.4.2.

Libraries used:

wheel==0.31.1
setuptools==39.1.0
twine==1.11.0

[matham]

I’ve also had a similar issue when trying to upload a sdist to PyPI that was rejected and twine check said:

Checking dist/Kivy-2.0.0rc1.tar.gz: FAILED
  `long_description` has syntax errors in markup and would not be rendered on PyPI.
    line 1: Severe: Unexpected section title or transition.

    ====
  warning: `long_description_content_type` missing.  defaulting to `text/x-rst`.

Even though we did set long_description_content_type='text/markdown'. However, I could only reproduce it when the sdist was generated on GitHub Action on a Windows CI instance, not locally.

Specifically, we set the long description by doing:

with open(join(dirname(__file__), 'README.md'), 'rb') as fileh:
    long_description = fileh.read().decode("utf8")

But the generated Windows CI generated sdist (with python setup.py sdist --formats=gztar) created a sdist whose PKG-INFO was malformed. Specifically, it’s Description field containing the description replaced all the \r\n in the original description with \r\r\n.

Changing our long_description to fileh.read().decode("utf8").replace('\r\n', '\n') fixed the issue.

[di]

Here’s another case of this happening: https://github.com/pypa/warehouse/issues/3664#issuecomment-439508132