What metadata/installability checks should Warehouse check uploads for?

Recently, a project that I worked on pushed a new version to PyPI and it turned out to be completely broken, because the setup.py was referencing a README.rst file that was not present in the sdist.

It would be awesome if PyPI could so checking of packages that are uploaded.

To start with, it could create a virtualenv and try pip installing the package and make sure that exits with status 0. Perhaps it could also try easy_install to make sure that works too.

There are more elaborate things that could be done like run tests if they are included (many packages don’t bundle their tests though) or try to validate the RST, but I think just pip installing is a very good first start, as it detects packages that are completely broken.

[di]

• Name is required and validated
• Version is required and validated
• Description is optional (and IMO should remain so)
• Author is optional (this field is also not required to contain contact information, so might not make sense to require)
• Classifiers is optional (and IMO should remain so, Requies-Python is probably what we should focus on here)

[msabramo]

Example of how CPAN runs tests on packages:

http://ppm4.activestate.com/i686-linux/5.16/1600/M/MS/MSABRAMO/App-TarColor-0.011.d/log-20120801T133902.txt

We can’t let Perl show us up! 😄

ActiveState is active in Python as well as Perl – maybe they are interesting in hosting a service like this like they do for the Perl community?