@grpc/grpc-js not working with aws ALB

Problem description

I was trying to use grpc with ALB, considering this documentation from AWS: https://aws.amazon.com/pt/blogs/aws/new-application-load-balancer-support-for-end-to-end-http-2-and-grpc/

I created the Load balancer, target group, cluster, service and task definition using an ECR image with this hello-world server example from grpc repo. I tried dynamic and static proto generation and both returned this error:

Error: 13 INTERNAL: Received RST_STREAM with code 2 triggered by internal client error: Protocol error
(node:1500234) UnhandledPromiseRejectionWarning: Error: 13 INTERNAL: Received RST_STREAM with code 2 triggered by internal client error: Protocol error
    at Object.callErrorFromStatus (/home/gustavosartori/validate-grpc-alb/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/home/gustavosartori/validate-grpc-alb/node_modules/@grpc/grpc-js/build/src/client.js:180:52)
    at Object.onReceiveStatus (/home/gustavosartori/validate-grpc-alb/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
    at Object.onReceiveStatus (/home/gustavosartori/validate-grpc-alb/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
    at /home/gustavosartori/validate-grpc-alb/node_modules/@grpc/grpc-js/build/src/call-stream.js:160:78
    at processTicksAndRejections (internal/process/task_queues.js:77:11)

After that to make sure the error wasn’t in my hello-world server I changed the ECR image to another grpc-server(using @grpc/grpc-jt) that I already have working on aws, and the error persists.

To finish I decided to test using the python hello world example, and it worked, so I believe the @grpc/grpc-js have some bad interaction with ALB, but I don’t know exactly how debug this

Reproduction steps

• Upload to ECR any grpc server (I used this hello-world example)
• Create a Cluster making sure to use an security group that allow traffic to the port used in the Dockerfile;
• Create a Target group: type = instance; protocol = http; port = 50051; protocol version = grpc;
• Create an Application Load Balancer configured to listen on 50051 and route to the just created target group;
• Create a task definition using the image uploaded to ECR
• Create a service to this task definition, in the load balancing section select the load balancer and target group created for the test
• Start de client using the dns of the ALB as url

Environment

• Fedora 35
• Node version 14.18.0]
• Node installation method: asdf
• Package name: “@grpc/grpc-js”: “^1.1.0”, “@grpc/proto-loader”: “^0.5.0”

Additional context

I tried generate a cert with openssl in order to no use grcp.createInsecure, but it changed nothin

[cabulafhy]

I’ve been looking into this with @g-sartori and we found the problem.

ALB ensures that only HTTPS listeners can foward requests to a gRPC target group, but we were consuming the server like it did’nt had any TLS/SSL in front of it. So we changed the way we created the client and it worked.

We were creating the client like this

var client = new hello_proto.Greeter(target, grpc.credentials.createInsecure())

And we changed to this

var client = new hello_proto.Greeter(target, grpc.credentials.createSsl())

In this documentation https://aws.amazon.com/pt/blogs/aws/new-application-load-balancer-support-for-end-to-end-http-2-and-grpc/ they do the same thing for the python hello-word example using credentials = grpc.ssl_channel_credentials()

[murgatroid99]

That capture shows that the server is responding with the error HTTP/1.1 400 Bad Request. Something is misconfigured and the front end your client is talking to isn’t handling HTTP/2. And the client doesn’t understand an HTTP/1.1 response to an HTTP/2 request, so that’s why the error is “Protocol error”.