Uncaught socket exception during timeout handling.
See original GitHub issueโ Iโm submitting a โฆ
- ๐ bug report
- ๐ฃ feature request
- โ question about the decisions made in the repository
๐ Describe the bug. What is the current behavior?
If a client goes away while a request timeout is being processed, the socket.error
exception produced in /cheroot/server.py
_conditional_error()
bubbles up uncaught all the way to the thread worker. This generates a stack trace regardless of logging config.
โ What is the motivation / use case for changing the behavior? This is producing STDERR errors in our container log, preventing us from handling errors properly. I also suspect given enough occurrence it may lead to thread pool starvation although I have not experimented with it.
๐ก To Reproduce This is hard to reproduce, I suspect timing is heavily involved. We are seeing it occur in our environment when requests timeout. These requests are coming from a load balancer.
๐ก Expected behavior The exception should be caught and treated as a simple failure and simply trigger completion and cleanup of the clientโs request (the client simply has gone away).
๐ Details Here is the traceback weโre seeing:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/local/lib/python2.7/site-packages/cheroot/workers/threadpool.py", line 114, in run
conn.communicate()
File "/usr/local/lib/python2.7/site-packages/cheroot/server.py", line 1286, in communicate
self._conditional_error(req, '408 Request Timeout')
File "/usr/local/lib/python2.7/site-packages/cheroot/server.py", line 1334, in _conditional_error
req.simple_response(response)
File "/usr/local/lib/python2.7/site-packages/cheroot/server.py", line 1115, in simple_response
self.conn.wfile.write(EMPTY.join(buf))
File "/usr/local/lib/python2.7/site-packages/cheroot/makefile.py", line 78, in write
data_mv[bytes_sent:bytes_sent + SOCK_WRITE_BLOCKSIZE],
File "/usr/local/lib/python2.7/site-packages/cheroot/ssl/pyopenssl.py", line 144, in send
*args, **kwargs
File "/usr/local/lib/python2.7/site-packages/cheroot/ssl/pyopenssl.py", line 91, in _safe_call
raise socket.error(errnum)
error: -1
๐ Environment
- PyOpenSSL version: 19.0.0
- Cheroot version: 6.5.4
- CherryPy version: 17.4.2
- Python version: 2.7.13
- OS: Linux Alpine
๐ Additional context
I believe a simple try/except/pass for the socket.error
in the context of https://github.com/cherrypy/cheroot/blob/v6.5.5/cheroot/server.py#L1276 should fix the issue without any negative side-effects since the server is already dealing with a client that has gone away.
Issue Analytics
- State:
- Created 4 years ago
- Comments:9 (5 by maintainers)
pyOpenSSL would also be latest pip, looks like 19.0.0.
Iโve reproduced this issue with a simple test case on cheroot 8.4.2 with pyOpenSSL 19.1.0. (You will need to replace ssl-cert.pem and ssl-cert-pk.pem below with your own certs, which can be self-signed). Let me know if I should open a separate issue rather than bumping this one.
Simple server-side code:
To reproduce:
Client-side steps to demonstrate problem: