Weird Errors when trying to use async reader with threading

I have a spider application which has to use 300+ concurrent consumers. I have no choice but to use async way of reader by following “Asynchronous example” from docs. My code is as follows: I use a cleanup_loop to check if thread is dead and make the message finish()

consume_tasks = []

def consume_func(content):
    # do custom stuff with content
    time.sleep(5)
    return

def async_handler(message):
    global consume_tasks
    message.enable_async()

    job = threading.Thread(
        target=consume_func,
        args=(message.body.decode(),)
    )
    consume_tasks.append({
        'message': message,
        'job': job,
    })
    job.start()

def cleanup_loop():
    global consume_tasks
    while True:
        unfinished_tasks = []
        for task in consume_tasks:
            if not task.get('job').is_alive():
                task.get('message').finish()
            else:
                unfinished_tasks.append(task)
        consume_tasks = unfinished_tasks
        time.sleep(0.1)

reader = nsq.Reader(message_handler=handler, lookupd_http_addresses='',
                    topic='Test123', channel='async', max_in_flight=10)
threading.Thread(target=cleanup_loop).start()
nsq.run()

I test with 20 concurrent producers pushing to this reader and it sometimes works (50%). Sometime is throwing me weird errors:

Traceback (most recent call last):
  File "/Users/tom/Virtualenvs/athena/lib/python3.5/site-packages/tornado/iostream.py", line 523, in _handle_events
    self._handle_write()
  File "/Users/tom/Virtualenvs/athena/lib/python3.5/site-packages/tornado/iostream.py", line 872, in _handle_write
    del self._write_buffer[:self._write_buffer_pos]
BufferError: Existing exports of data: object cannot be re-sized

Traceback (most recent call last):
  File "/Users/tom/Virtualenvs/athena/lib/python3.5/site-packages/tornado/ioloop.py", line 888, in start
    handler_func(fd_obj, events)
  File "/Users/tom/Virtualenvs/athena/lib/python3.5/site-packages/tornado/stack_context.py", line 277, in null_wrapper
    return fn(*args, **kwargs)
  File "/Users/tom/Virtualenvs/athena/lib/python3.5/site-packages/tornado/iostream.py", line 523, in _handle_events
    self._handle_write()
  File "/Users/tom/Virtualenvs/athena/lib/python3.5/site-packages/tornado/iostream.py", line 847, in _handle_write
    assert self._write_buffer_size >= 0
AssertionError

and then

ERROR:nsq.client:[127.0.0.1:4150:athena.Test123:async] ERROR: SendError: failed to send FIN b'0c18c9d08fd9d001' (Stream is closed)

From full day of debugging, I know this error is coming from the line task.get('message').finish() but I don’t know why.

Can anyone help me with this? Any advice is welcome! If this is not going good with Threading, what should I use instead?

[ploxiln]

def process_thread(msg_body):
    # do stuff with msg body
    time.sleep(5)

def handler(message):
    message.enable_async()

    def finish():
        message.finish()

    loop = tornado.ioloop.IOLoop.current()
    fut = loop.run_in_executor(None, process_thread, message.body)
    loop.add_future(fut, finish)

[lambdaq]

getting the same error with pynsq==0.9.0

No threading, just plain old single threaded nsq.