Twisted 19.2.0 assumes http headers are of string type

After trying to run scrapyd on both MacOS Mojave, and a fresh AWS instance based on AMI ami-0ff760d16d9497662, I’m always hitting the same error.

Steps are as follows:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python get-pip.py
sudo yum install gcc
sudo yum install python-devel
sudo pip install scrapyd
vi scrapyd.conf
scrapyd

The contents of scrapyd.conf are that of the example, with the addition of a different bind_address:

[scrapyd]
eggs_dir    = eggs
logs_dir    = logs
items_dir   =
jobs_to_keep = 5
dbs_dir     = dbs
max_proc    = 0
max_proc_per_cpu = 4
finished_to_keep = 100
poll_interval = 5.0
http_port   = 6800
debug       = off
runner      = scrapyd.runner
application = scrapyd.app.application
launcher    = scrapyd.launcher.Launcher
webroot     = scrapyd.website.Root
bind_address = 0.0.0.0

[services]
schedule.json     = scrapyd.webservice.Schedule
cancel.json       = scrapyd.webservice.Cancel
addversion.json   = scrapyd.webservice.AddVersion
listprojects.json = scrapyd.webservice.ListProjects
listversions.json = scrapyd.webservice.ListVersions
listspiders.json  = scrapyd.webservice.ListSpiders
delproject.json   = scrapyd.webservice.DeleteProject
delversion.json   = scrapyd.webservice.DeleteVersion
listjobs.json     = scrapyd.webservice.ListJobs
daemonstatus.json = scrapyd.webservice.DaemonStatus

However, when trying to run scrapyd-deploy, or even browse to :6800/jobs, The following error is thrown:

2019-04-11T08:55:33+0000 [twisted.python.log#info] "_redacted_" - - [11/Apr/2019:08:55:33 +0000] "GET /jobs HTTP/1.1" 500 6077 "http://_redacted_:6800/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"
2019-04-11T08:55:33+0000 [twisted.web.server.Request#critical]
	Traceback (most recent call last):
	  File "/usr/lib64/python2.7/site-packages/twisted/protocols/basic.py", line 572, in dataReceived
	    why = self.lineReceived(line)
	  File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line 2105, in lineReceived
	    self.allContentReceived()
	  File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line 2196, in allContentReceived
	    req.requestReceived(command, path, version)
	  File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line 920, in requestReceived
	    self.process()
	--- <exception caught here> ---
	  File "/usr/lib64/python2.7/site-packages/twisted/web/server.py", line 199, in process
	    self.render(resrc)
	  File "/usr/lib64/python2.7/site-packages/twisted/web/server.py", line 259, in render
	    body = resrc.render(self)
	  File "/usr/lib/python2.7/site-packages/scrapyd/website.py", line 157, in render
	    txrequest.setHeader('Content-Length', len(s))
	  File "/usr/lib64/python2.7/site-packages/twisted/web/http.py", line 1271, in setHeader
	    self.responseHeaders.setRawHeaders(name, [value])
	  File "/usr/lib64/python2.7/site-packages/twisted/web/http_headers.py", line 220, in setRawHeaders
	    for v in self._encodeValues(values)]
	  File "/usr/lib64/python2.7/site-packages/twisted/web/http_headers.py", line 40, in _sanitizeLinearWhitespace
	    return b' '.join(headerComponent.splitlines())
	exceptions.AttributeError: 'int' object has no attribute 'splitlines'

Some versions:

# python --version
Python 2.7.5
# twistd --version
twistd (the Twisted daemon) 19.2.0
Copyright (c) 2001-2019 Twisted Matrix Laboratories.
See LICENSE for details.

[hellobiek]

downgrade Twisted to verison 18.9.0 can fix this problem.

pip3 install Twisted==18.9.0

[patalanov]

Everything works fine now. Bug is gone, data crawled and parsed. Thank you for being so helpful and kind, @my8100