CI should fail on deprecation warnings

It’s hard for us to detect when we are using deprecated code, which allows things like #1798 to go undetected for a long time. In dateutil I treat all warnings as errors so that we can actually heed the warnings that are being raised by my upstream dependencies.

I think we should do the same thing here. I tried enabling warnings-as-errors on #1798 and it actually worked for a few days, then bdist_wininst was deprecated breaking us on the dev build of CPython. I think we should remove our use of bdist_wininst or suppress the warning then enable warnings-as-errors.

[jaraco]

I basically agree with all of that. Happy to enforce no warnings. Even better is to have a clear guidance on how to suppress errors/warnings when the emerge and to file a corresponding ticket to address the deprecation. Perhaps that guidance could go here.

I imagine something like:

When encountering an error from a Warning:

1. File a ticket including the text from the failure.
2. Suppress the warning by adding an entry to pytest.ini, including a link to the ticket.

Can we test a process like that with bdist_wininst?

[jaraco]

Enabling this feature is going to cause a lot of toil on this project. Every time a deprecation emerges, it’s going to be blocked by this feature. Case in point #2016. That’s not too bad because it means that a ticket can be filed and the error can be suppressed until the issue can be addressed (if not addressed immediately), but it’s going to require additional resources to keep up with these issues in a timely manner to prevent them from blocking otherwise good pull requests.