Return more detailed error message in scrapy check

When I run scrapy check myspider, it returns an error message of

Unhandled error in Deferred:


----------------------------------------------------------------------
Ran 0 contracts in 0.000s

OK

However, if I run scrapy crawl myspider, it returns an error message of

2019-04-02 16:37:21 [twisted] CRITICAL: Unhandled error in Deferred:

2019-04-02 16:37:21 [twisted] CRITICAL: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python3.6/site-packages/scrapy/crawler.py", line 82, in crawl
    yield self.engine.open_spider(self.spider, start_requests)
pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 61] Connection refused

In the end, it was the database connection that is causing the issue, but in scrapy check it doesn’t return anything useful for me to debug. I think we should have more details in the error message.

If you guys think this is a good improvement, I will work on this as my first issue in this project. Thanks!

[Gallaecio]

Not really. I believe SPIDER_CONTRACTS is only necessary for custom contracts, builtin contracts require only to be specified in the docstring of a callback.

[herodionem]

I’m still new to this so someone please correct me if I’m wrong but I believe, at least in this case, you aren’t ever providing any contracts to be checked. ITEM_PIPELINES are executed during a scrapy crawl after an Item is yielded from the parse function. If you are wanting to execute a contract, you will need to enable SPIDER_CONTRACTS = { 'checker.contracts.SomeContract' } in your settings and make sure you have a Contract sub-class named SomeContract in the ./checker/contracts.py file.