Unexpected exception raised in websocket.

I am running hasura/graphql-engine:v1.3.1 docker image on an ec2 t2.medium instance. Sometime i encounter sequence of following errors in the logs. I am using apollo client for subscriptions.

{"type":"unstructured","timestamp":"2020-10-19T14:41:01.279+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"} {"type":"unstructured","timestamp":"2020-10-19T14:43:36.701+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"} {"type":"unstructured","timestamp":"2020-10-19T14:44:31.333+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"} {"type":"unstructured","timestamp":"2020-10-19T14:45:01.085+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"} {"type":"unstructured","timestamp":"2020-10-19T14:48:00.665+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"} {"type":"unstructured","timestamp":"2020-10-19T14:48:00.665+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"} {"type":"unstructured","timestamp":"2020-10-19T14:51:01.300+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}

[24601]

This continues to be an issue for us. In this day and age, 5MB of data is “not that much” and a system really should not croak on it. If PG can handle it, the graphql-engine on top of it should. Graphql-engine should not impose limitations on size that are smaller than those handled by PG/underlying DB…that seems anathema to the whole point of the engine…

Even so, killing the thread (which borks tons of other stuff) is not a great way to handle this even if this limitation exists…

Happens as recently as 2.6.2.

#6542 seems to have same issue.

[karibertils]

{"type":"unstructured","timestamp":"2022-01-30T15:56:58.584+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T15:56:58.584+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T15:56:58.584+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T15:56:58.584+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T15:56:58.584+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T15:56:58.584+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}

{"type":"unstructured","timestamp":"2022-01-30T16:10:54.411+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:10:54.411+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:12:36.807+0000","level":"warn","detail":"Overdue for a major GC: forcing one even though we don't appear to be idle"}

Also

{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Thread killed by timeout manager"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}
{"type":"unstructured","timestamp":"2022-01-30T16:18:35.944+0000","level":"error","detail":"Unexpected exception raised in websocket. Please report this as a bug: Warp: Client closed connection prematurely"}

Getting this all the time. I’m running hasura/graphql-engine:v2.1.1

Currently, I have a configuration where this happens every time with graphql engine running for like 2-3 minutes. Seems to be somehow related to the clients which have active subscriptions.