Multipart upload splits CRLF across 2 chunks

Expected behavior

CLRF is always part of a single chunk.

Actual behavior

CRLF can be split across 2 chunks (in some super rare exotic circumstances), causing the first CR to be considered part of the multipart payload and being offered to the Decoder, causing

io.netty.handler.codec.http.multipart.HttpPostRequestDecoder$ErrorDataDecoderException: java.io.IOException: Out of size: 5043445 > 5043444
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.loadDataMultipartOptimized(HttpPostMultipartRequestDecoder.java:1190)
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.getFileUpload(HttpPostMultipartRequestDecoder.java:926)
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.decodeMultipart(HttpPostMultipartRequestDecoder.java:572)
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.parseBodyMultipart(HttpPostMultipartRequestDecoder.java:463)
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.parseBody(HttpPostMultipartRequestDecoder.java:432)
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.offer(HttpPostMultipartRequestDecoder.java:347)
	at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.offer(HttpPostMultipartRequestDecoder.java:54)
	at io.netty.handler.codec.http.multipart.HttpPostRequestDecoder.offer(HttpPostRequestDecoder.java:223)

This is very much a theory. Any hints to figure out whether something else could cause this would be appreciated.

Steps to reproduce

We have a test in Apache Flink that uploads a multipart HTTP request containing a bit of JSON as a MemoryAttribute and a FileUpload. When the size of these attributes is just right (deviating by as much as 3 bytes hides the issue) the decoding of the request fails. If it fails, then it is always with an off-by-one error as shown above, one more byte arriving than expected.

While debugging I inspected the chunk that causes the error. The last readable byte in the chunk had the value ‘13’, i.e., CR. This makes me to believe that it is somehow possible that the CRLF can be split across chunks.

According to the the 4.1.49 decoder this should be impossible, but the explicit check was removed in 1529ef1794e0a6654fa4334fd979b769d6940e61.

We further consistently observe that the return value HttpPostRequestEncoder#length() after finalizing the request differs by exactly 1 between successful and failed runs, e.g., 5043832 (successful) vs 5043831 (failed). Not sure what to make of that, but it’s too consistent to ignore.

We have only observed this error after upgrading from 4.1.49 to 4.1.65.

Minimal yet complete reproducer code (or URL to code)

Still working on that. I could provide a reproducer that relies on a few Apache Flink classes if that would be acceptable.

Netty version

4.1.65.Final

JVM version (e.g. java -version)

openjdk version “11” 2018-09-25 OpenJDK Runtime Environment 18.9 (build 11+28) OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)

OS version (e.g. uname -a)

Windows 11, but we also observed it on Ubuntu 20.

[fredericBregier]

IMHO, it is the contrary:

• when the CR is there but not LF, then it returns -1 since it does not found the boundary ending
• when it gets the next buffer, then LF will be found However, I think I can do a reproducer as you described (very easy). It will then show the bug, if any, and finally I will propose a fix (of course, if I found the root cause).

[fredericBregier]

OK, thanks ! I’m trying to find out the best way to fix it (with no new issue of course)…