Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

can't export to json file when use JsonWriterPipeline

See original GitHub issue

Scrapy 1.0.5 Python 2.7.11

pipelines.py

# -*- coding: utf-8 -*-

# Define your item pipelines here
#
# Don't forget to add your pipeline to the ITEM_PIPELINES setting
# See: http://doc.scrapy.org/en/latest/topics/item-pipeline.html

import codecs
import json


class JsonWriterPipeline(object):
    def __init__(self):
        self.file = codecs.open('items.json', mode = 'wb', encoding = 'utf-8')

    def process_item(self, item, spider):
        line = json.dumps(dict(item)) + "\n"
        self.file.write(line.decode('unicode_escape'))
        return item

Error information

2016-03-16 00:03:39 [scrapy] ERROR: Error processing {'records': [{'en_kicker': u'Sinosphere'}]}
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/Users/linzhen/OneDrive/Source/newsCrawler/newsCrawler/pipelines.py", line 17, in process_item
    line = json.dumps(dict(item)) + "\n"
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 244, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File 
![qq20160316-0 2x](https://cloud.githubusercontent.com/assets/4090783/13784812/a6394e30-eb0b-11e5-9753-4fb1bfd3530b.png)
"/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: {'en_kicker': u'Sinosphere'} is not JSON serializable

item variables