logging in pipelines breaks scrapy deploy success message

Not sure if this should be reported here on on scrapyd’s issue tracker.

using the logging facility on scrapy breaks the spider count on deploy, it returns 0 all the time.

To reproduce the error, place the snippet below on your pipelines module and try to deploy to a scrapyd target

from scrapy import log

class MyPipeline(object):    
    def __init__(self):
        log.start()

...

[stav]

Just use the spider’s built-in logging:

class MyPipeline(object):
    def process_item(self, item, spider):
        spider.log('Well, here is an Item: %s.' % item)

[JamesTheHacker]

@pablohoffman I came here because I’m had the same question as OP. To answer the question about why do it in a pipeline … in my case it’s because I’m storing items in a MySQL database from within the pipeline. I need to log any exceptions when inserting into the database.

In my case I needed to log an ERROR. This is how I did it:

import logging
spider.log('Error writing to database: {0}'.format(e), logging.ERROR)