boto3 release process on PyPI is broken

https://pypi.org/pypi/boto3/1.9.220/json doesn’t correctly list botocore as a dependency. As a result https://libraries.io/pypi/boto3 and other tools are unable to detect those dependencies correctly.

[swetashre]

@techtonik - Thanks for sharing your output. Our release process first uploads the source package to pypi then the binary package. After uploading the source package pypi does not look for any dependency so evenif the binary package has all the dependency it never gets updated.

We could first publish the binary package as you suggested to update all those dependency. I would mark this as enhancement.

[techtonik]

Hi @swetashre. Does AWS have any kind of invisible infrastructure support program like https://gitcoin.co or Tidelift? Over the last two days I tracked the problem down to the following issues:

1. https://pypi.org (warehouse) reads dependencies from source and binary packages, but source package for boto3 doesn’t contain dependency metadata (2). Luckily, binary package for boto3 contains the correct metadata. https://pypy.org doesn’t have any logic to ensure that dependencies are filled correctly - the first uploaded package wins.

quick hack solution for boto3 to publish its dependencies is to upload binary package first. The release process is not documented, so I couldn’t send a PR to fix that.

proper solution part 1 is to patch https://pypi.org to fill missing metadata from subsequently uploaded packages with the same version.

1. Another way to fix things is to make sure that dependency metadata is included in source package of boto3. If you’re using python setup.py dist to create the package, then setuptools is the tool that needs to be patched.

proper solution part 2 is to implement missing part in setuptools.