Server not accepting requests after update from 1.41.0 to 1.42.0

After upgrading from 1.41.0 to 1.42.0, our services stopped receiving requests, and logging following INFO message:

• [id: 0x84be72e8, L:/172.21.223.199:8080 - R:/172.21.225.64:58430] ignoring DATA frame for stream RST_STREAM sent.
• sourceClassName: io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener
• sourceFileName: defaulthttp2connectiondecoder.java
• sourceMethodName: shouldIgnoreHeadersOrDataFrame
• sourceLineNumber: 648

We think that it could be caused by the following changes:

• netty: Requests with Connection header are malformed
• dep: bump netty to 4.1.63.Final and tcnative to 2.0.38.Final

As we are running our workload also in a legacy environment where proxies don’t handle HTTP/2 100% correctly, the issue is probably in the Connection header.

Is your feature request related to a problem?

Connections with the Connection header are refused.

Describe the solution you’d like

Make this validation configurable, so have a new method in ServerBuilder class 🙏

Describe alternatives you’ve considered

Remove this validation completely.

Additional context

Unfortunately, this is a critical issue for us that blocks us from updating gRPC libraries 😢

My guess why netty doesn’t reject the connection with the Connection header is that there are still too many legacy workloads that could be broken by this change.

[ejona86]

We can provide a system property or environment variable to disable the new validation. And then we just coordinate to determine a reasonable amount of time to leave it in place.

[ejona86]

The system property io.grpc.netty.disableConnectionHeaderCheck will be available. Note that it will be io.grpc.netty.shaded.io.grpc.netty.disableConnectionHeaderCheck for grpc-netty-shaded.