Should we delete the deprecated code in Scrapy 2.0?

I recopilated some of the old (before 2018) deprecation warnings to discuss if we can delete them in the upcoming Scrapy 2.0 release (chronological order):

BaseSgmlLinkExtractor and SgmlLinkExtractor (classes)

• Deprecation warnings: 07/2014
• Links:
  • https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/linkextractors/sgml.py#L23
  • https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/linkextractors/sgml.py#L116
• PR: #4679

Crawler.spiders (property)

• Deprecation warning: 08/2014
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/crawler.py#L74
• PR: https://github.com/scrapy/scrapy/pull/4398

HtmlParserLinkExtractor (class)

• Deprecation warning: 05/2015
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/linkextractors/htmlparser.py#L22
• PR: #4674

stringify_dict (function)

• Deprecation warning: 07/2015
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/utils/python.py#L296
• PR: https://github.com/scrapy/scrapy/pull/4362

is_writable (function)

• Deprecation warning: 07/2015
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/utils/python.py#L311
• PR: https://github.com/scrapy/scrapy/pull/4362

setattr_default (function)

• Deprecation warning: 07/2015
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/utils/python.py#L322
• PR: https://github.com/scrapy/scrapy/pull/4362

Initialising scrapy.extensions.feedexport.S3FeedStorage without AWS keys is deprecated. (procedure)

• Deprecation warning: 11/2015
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/extensions/feedexport.py#L103
• PR: https://github.com/scrapy/scrapy/pull/4411

scrapy.utils.datatypes.MultiValueDict, MultiValueDictKeyError and SiteNode (classes)

• Deprecation warnings: 01/2016
• Links:
  • https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/utils/datatypes.py#L46
  • https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/utils/datatypes.py#L19
  • https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/utils/datatypes.py#L161
• PR: https://github.com/scrapy/scrapy/pull/4400/

isbinarytext (function)

• Deprecation warning: 03/2016
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/utils/python.py#L168
• PR: https://github.com/scrapy/scrapy/pull/4362

scrapy.downloadermiddlewares.chunked (module)

• Deprecation warning: 11/2016
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/downloadermiddlewares/chunked.py#L7
• PR: https://github.com/scrapy/scrapy/pull/4431/

Spider.make_requests_from_url (method)

• Deprecation warning: 02/2017
• Link: https://github.com/scrapy/scrapy/blob/c4ee4b6075fbb616272788b14b70f0af63dc2877/scrapy/spiders/init.py#L66
• PR: https://github.com/scrapy/scrapy/pull/4178

There are other deprecation warnings but as they are from 2018 to the present I think we can leave it there for a while. However, as we are going to release a major version maybe is a good moment to do it.

Do you see any inconvenience?

Any thoughts? Let me know what you think.

[noviluni]

I removed the “good first issue” label to avoid similar future questions.

[noviluni]

Hi @nyov @elacuesta @Gallaecio @kmike , thank you for taking this!! 🚀

I updated the issue description with the PRs.