Long polling cripples cherrypy

Originally reported by: shapeshifter (Bitbucket: shapeshifter, GitHub: @shapeshifter)

---

I’m having trouble making long polling work correctly with cherrypy:

Here’s a test case. There’s a boolean “self.done” which can be switched using “/set_done” and “/set_pending”. While it’s False, “/long_poll” will be kept open and when calling “/set_done”, it will return a response saying “Done!”.

The problem is: When calling long_poll a couple of times (about 5-10), the server becomes very unresponsive. The problem is even more distinct when killing the long_poll connections from the client. In that case, the server becomes completely unresponsive. This happens even though I’ve set the thread pool limit to -1.

cherrypy_config:

[global]
tree.test = test.app
server.socket_host = '0.0.0.0'
server.socket_port = 8090
server.thread_pool_max = -1
tools.encode.encoding = 'utf-8'
tools.encode.on = True 
log.error_file = "/tmp/cp_errors.log"

[/]
tools.sessions.on = False

test.py:

#!/usr/bin/python2
# -*- coding: utf-8 -*-

import time
import cherrypy

class Test(object):
    def __init__(self):
        self.done = False

    @cherrypy.expose
    def long_poll(self):
        cherrypy.response.timeout = 7200
        while True:
            if self.done == False:
                time.sleep(0.5)
            else:
                return "Done!\n"

    @cherrypy.expose
    def set_done(self):
        self.done = True
        return "set done\n"

    @cherrypy.expose
    def set_pending(self):
        self.done = False
        return "set pending\n"

    @cherrypy.expose
    def test(self):
        return "still working OK\n"

app = cherrypy.Application(Test())

Here’s a sample run using just wget.

Start the server:

[straydog@saskatoon testcase]$ cherryd -c cherrypy_config
[07/Jan/2012:15:16:25] ENGINE Mounted: cherrypy._cptree.Application(<test.Test object at 0x96b6d8c>, '') on /
[07/Jan/2012:15:16:25] ENGINE Listening for SIGHUP.
[07/Jan/2012:15:16:25] ENGINE Listening for SIGTERM.
[07/Jan/2012:15:16:25] ENGINE Listening for SIGUSR1.
[07/Jan/2012:15:16:25] ENGINE Bus STARTING
[07/Jan/2012:15:16:25] ENGINE Started monitor thread '_TimeoutMonitor'.
[07/Jan/2012:15:16:25] ENGINE Started monitor thread 'Autoreloader'.
[07/Jan/2012:15:16:25] ENGINE Serving on 0.0.0.0:8090
[07/Jan/2012:15:16:25] ENGINE Bus STARTED

Initial test:

[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/test -O -
still working OK

Start one long_poll in the background to demo the functionality and then call set_done. Everything works as expected:

[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[1] 1794
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/set_done -O -
set done
[shapeshifter@bluerabbit cherrypylongpolltest]$ Done!

[1]+  Done                    wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O -

Set pending again, and then open several long_poll connections in background:

[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/set_pending -O -
set pending
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/test -O -
still working OK
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[1] 1798
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[2] 1799
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[3] 1800
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[4] 1801
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[5] 1802
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[6] 1803
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[7] 1804
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[8] 1805
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[9] 1806
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[10] 1807
[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/long_poll -O - &
[11] 1808

Now calling even just “test” which should just return a message hangs forever:

[shapeshifter@bluerabbit cherrypylongpolltest]$ wget -q http://saskatoon.icu.uzh.ch:8090/test -O -
#not working anymore!

At this point, even “killall wget” doesn’t help and if anything, makes the problem worse.

I have no idea how to remedy this or if this behavior is to be expected but with a -1 thread pool, I’d guess that cherrypy should always be able to answer requests, even with several open connections.

---

• Bitbucket: https://bitbucket.org/cherrypy/cherrypy/issue/1120

[jaraco]

See cherrypy-dynpool for a project that implements a dynamic thread pool. Contributions to improve the implementation or have the documentation match the implementation are also welcome.

[clawsoon]

@webknjaz I’m not sure if I’m smart enough. 😃