UNAVAILABLE: HTTP/2 error code: NO_ERROR Received Goaway

What version of gRPC-Java are you using?

1.30.2

What is your environment?

Linux xxxxx016 3.10.0-1062.18.1.el7.x86_64 #1 SMP Tue Mar 17 23:49:17 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux java version “1.8.0_91” Java™ SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot™ 64-Bit Server VM (build 25.91-b14, mixed mode)

What did you expect to see?

What did you see instead?

 io.grpc.StatusRuntimeException: UNAVAILABLE: HTTP/2 error code: NO_ERROR
Received Goaway
	at io.grpc.Status.asRuntimeException(Status.java:533)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:460)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
	at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:751)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:740)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Steps to reproduce the bug

The java grpc client connects to the go grpc service (service grpc version 1.27.1). The above exception occurs. The service uses container deployment and nginx for load balancing. There is a high probability of this abnormal situation. Of course, we are also working hard to find a solution, but there is no progress. Hope to get your guidance and help. I will be very grateful.

[ejona86]

I think we can change the workaround to only trigger if the HTTP/2 error-code is indicates an error (anything but NO_ERROR). That would workaround the Netty problem but allow transparent retries to function on servers that use abrupt GOAWAYs.

[ejona86]

Filed https://trac.nginx.org/nginx/ticket/2224 for nginx’s lack of double-GOAWAY.