Unhandled OpenSSL.SSL.WantReadError and WantWriteError raised by PyOpenSSL

❓ I’m submitting a …

• 🐞 bug report
• 🐣 feature request
• ❓ question about the decisions made in the repository

🐞 Describe the bug. What is the current behavior? This issue is reproducible against huge post request like multiple lines of textarea in form. We are upgrading from python 2.7 to 3.7 and hence also upgraded the cheroot 8.2.0. Cherrypy : 18+

📋 Details

Traceback (most recent call last):
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 628, in respond
    self._do_respond(path_info)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 680, in _do_respond
    self.body.process()
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 982, in process
    super(RequestBody, self).process()
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 559, in process
    proc(self)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 145, in process_urlencoded
    qs = entity.fp.read()
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 817, in read
    data = self.fp.read(chunksize)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cheroot/server.py", line 383, in read
    data = self.rfile.read(size)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/cheroot/makefile.py", line 420, in read
    val = super().read(*args, **kwargs)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1014, in read
    return self._read_unlocked(size)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/_pyio.py", line 1054, in _read_unlocked
    chunk = self.raw.read(wanted)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1335, in recv_into
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/pyenv/versions/3.7.0/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1149, in _raise_ssl_error
    raise WantReadError()
OpenSSL.SSL.WantReadError

📋 Environment

• Cheroot version: 8.2.0
• CherryPy version: 18.1.1 (if applicable)
• Python version: 3.6.7
• OS: linux
• Browser: [all | Chrome XX | Firefox XX | IE XX | Safari XX | Mobile Chrome XX | Android X.X Web Browser | iOS XX Safari | iOS XX UIWebView | iOS XX WKWebView ]

📋 Additional context I have went through this https://github.com/cherrypy/cheroot/issues/113 but still was unable to figure out the solution

[Rohitbels]

@jpeckham Thanks a lot for this!

@webknjaz and for any one else:

I was able to fix this by replacing to read call at

File "/cheroot/makefile.py", line 420, in read val = super().read(*args, **kwargs)

with

val = self._safe_call(True, super().read, *args, **kwargs)

safe_call is specifically defined in pyopenssl.py for this reason.

@webknjaz This fix looks good?

[jpeckham]

I was getting WantWriteError on very large js files and we did this monkey patch

def _flush_unlocked(self):
    self._checkClosed('flush of closed file')
    while self._write_buf:
        try:
            # ssl sockets only except 'bytes', not bytearrays
            # so perhaps we should conditionally wrap this for perf?
            n = self.raw.write(self._write_buf)
        except io.BlockingIOError as e:
            n = e.characters_written
        except (OpenSSL.SSL.WantReadError,OpenSSL.SSL.WantWriteError, OpenSSL.SSL.WantX509LookupError) as e:
            n = 0
        del self._write_buf[:n]

BufferedWriter._flush_unlocked = _flush_unlocked