Occasional GRPC crashes with testnet transactions

Problem

“@onflow/fcl”: “1.0.2”, “@onflow/transport-grpc”: “1.0.1”

Hi, this is Todd at GigLabs. When executing transactions on testnet via FCL,we are getting intermittent errors. Not all the time, but enough that it’s causing problems for our Ops and QA teams trying to test things. No reports so far of this on mainnet. All of these calls are in a try/catch block in our code, but it seems like our node process experiences a hard crash in sdk-send-grpc.js. Can you help me figure out 1) the cause of this error and 2) how the exception can be caught? Right now we have no visibility into what happened and we pretty much have to restart our server. Presumably this would allow our app to access statusMessage. I have tried in the past to switch from GRPC to HTTP, but that didn’t allow me to query events reliably (other random errors). So we are still using GRPC. Thank you!

/usr/src/portal-api/node_modules/@onflow/transport-grpc/dist/sdk-send-grpc.js:24
2022-09-15 12:19:13reject(new Error(statusMessage));
2022-09-15 12:19:13^
2022-09-15 12:19:13Error
2022-09-15 12:19:13at Object.onEnd (/usr/src/portal-api/node_modules/@onflow/transport-grpc/dist/sdk-send-grpc.js:24:22)
2022-09-15 12:19:13at /usr/src/portal-api/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:24233
2022-09-15 12:19:13at /usr/src/portal-api/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:11490
2022-09-15 12:19:13at Array.forEach (<anonymous>)
2022-09-15 12:19:13at e.rawOnError (/usr/src/portal-api/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:11452)
2022-09-15 12:19:13at e.onTransportHeaders (/usr/src/portal-api/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:9025)
2022-09-15 12:19:13at NodeHttp.responseCallback (/usr/src/portal-api/node_modules/@improbable-eng/grpc-web-node-http-transport/lib/index.js:31:22)
2022-09-15 12:19:13at Object.onceWrapper (node:events:514:26)
2022-09-15 12:19:13at ClientRequest.emit (node:events:394:28)
2022-09-15 12:19:13at ClientRequest.emit (node:domain:470:12)
2022-09-15 12:19:13at HTTPParser.parserOnIncomingClient (node:_http_client:621:27)
2022-09-15 12:19:13at HTTPParser.parserOnHeadersComplete (node:_http_common:127:17)
2022-09-15 12:19:13at TLSSocket.socketOnData (node:_http_client:487:22)
2022-09-15 12:19:13at TLSSocket.emit (node:events:394:28)
2022-09-15 12:19:13at TLSSocket.emit (node:domain:470:12)
2022-09-15 12:19:13at addChunk (node:internal/streams/readable:312:12)

Steps to Reproduce

Happens randomly when doing things like minting NFTs on testnet. Seems to be more frequent when running multiple transactions in parallel with different keys.

Context

It becomes a dev ops problem for us, and it blocks our ops and QA teams from doing their jobs on testnet in preparation for production NFT drops.

[bluesign]

yeah 40 shouldn’t be a problem actually, maybe you can try updating the FCL. I remember some exception problems talk before, but it was about http api part mostly.

[gta14]

@peterargue We are sticking with grpc for now, but if we change I will let you know if there are any issues. When I tried HTTP before we weren’t doing anything special, just querying events from NFT contracts.