question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

WebSockets: No timeout associated to Ping-Pong-Frames

See original GitHub issue

Describe the bug

The Websocket RFC defines in Section 5.5.2 and 5.5.3 the Ping and Pong frames. ASP.net Core 3.1 is sending a Ping-Frame from the Server to the client in the defined interval from WebSocketOptions.KeepAliveInterval property. So far everything works as expected.

But with a “malicious” client, which is not responding with a mandatory Pong-Frame, the Server just keeps sending Ping-Frames and nothing happens.

After quickly scanning through the repo, i am afraid that there is simply nothing implemented, that checks if the client responds with a pong frame in time. To make this clear: The RFC is not defining what the other side should do, when Pong-Fames are not coming back. But it clearly states that the receiver of a Ping-Frame MUST send a Pong-Frame back. Based on that i was expecting some server-side timeout for the websocket connection (like missing three Pong-Frames terminates the connection).

This behavior currently prevents the server from detecting broken Websocket-Connections where e.G. an intermediate Router has gone away and now WebSocket Close-Frame and/or TCP FIN packet has been transmitted.

To Reproduce

This is not that easily reproducable, since you need a WebSocket-Client which you can “freeze” without sending a Close-Frame to the server. I used Mojo::UserAgent in a perl script to connect a websocket and after a while i paused the process via CTRL+Z on the command line.

I watched the network traffic via Wireshark and verified that Ping-Frames are sent and no Pong-Frames are going back anymore. The red line marks the moment when i froze the client-websocket.

https://sveneppler.de/stuff/asp-net-websocket-ping-pong.png

I could provide a reproducer repo. But since to my knowledge there is just no handling in the server anyway, a reproducer would be “useless”.

Further technical details

  • ASP.NET Core 3.1
  • Include the output of dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.301
 Commit:    7feb845744

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  18.04
 OS Platform: Linux
 RID:         ubuntu.18.04-x64
 Base Path:   /usr/share/dotnet/sdk/3.1.301/

Host (useful for support):
  Version: 3.1.5
  Commit:  65cd789777

.NET Core SDKs installed:
  2.1.807 [/usr/share/dotnet/sdk]
  2.2.402 [/usr/share/dotnet/sdk]
  3.1.301 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.19 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.19 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.5 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.19 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.5 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:8
  • Comments:9 (3 by maintainers)

github_iconTop GitHub Comments

3reactions
bjornen77commented, Feb 25, 2021

I think that adding a WebSocketOptions.Timeout property would be a good thing. As @BrennanConroy says above, the WebSocket is only closed when the TCP Retransmission times out. This timeout seems to be different on Windows and Linux. By default, on Windows it takes about 21 seconds and on Linux it takes about 15 minutes before the WebSocket is closed.

The ClientWebSocket today only sends “Pong” to the server, and therefore it does not wait for a “Pong” response to arrive. I think that the ClientWebSocket shall send a “Ping” instead, and then verify that a “Pong” is received within the Timeout set. If not received, the connection shall be closed. Or at least it should expose the "received “Pong” message in the WebSocketReceiveResult.MessageType. It would be good if an received “Ping” message from a server is also exposed. Then the application could use these Ping/Pong message types to implement its own timeout logic.

1reaction
osexpertcommented, Sep 21, 2020

As being resolved duplicate from #26117 I add a summary here: It is easy to reproduce, pull out network cable:-) ClientWebSocket does timeout (and throws WebSocketException) after not recieving Ping or Pong. Inconsistent. RFC does not say anything about this either, so why not let the client hang forever too:-D But seriously, RFC only says Pingpong can be used as a tool to implement heartbeat\timeout for endpoints (eg. both parties): “NOTE: A Ping frame may serve either as a keepalive or as a means to verify that the remote endpoint is still responsive.” You may choose to not use the tools given, and it will work poorly. You may use them and it will work good. The RFC does not care, we do:-)

Here is exception from client side:

System.Net.WebSockets.WebSocketException (0x80004005): The remote party closed the WebSocket connection without completing the close handshake.
 ---> System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
 ---> System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.ReadBufferedAsyncCore(Memory`1 destination)
   at System.Net.Http.HttpConnection.RawConnectionStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure)
   at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory`1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter)
   at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory`1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter)

It does not seem related to Pingpong, possibly the client dies because of other circumstances. By coincidence the server does not die in the same way. I guess I come to same conclusion as original poster, that .Net WebSocket does not implement any of timeout related to Pingpong.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Sending websocket ping/pong frame from browser
I keep reading about ping/pong messages in websockets to keep the connection alive, but I'm not sure what they are. Is it a...
Read more >
Timeouts - websockets 11.0.3 documentation
If the Pong frame isn't received, websockets considers the connection broken and closes it. This mechanism serves two purposes: It creates a trickle...
Read more >
Speech to Text WebSocket interface
If your WebSocket stack does not implement ping-pong frames and your are sending long audio files, your connection can experience a read timeout....
Read more >
Ballerina API Docs - websocket
If the timeout exceeds, then the connection is terminated even though a close frame is not received from the remote endpoint. If the...
Read more >
WebSockets support in ASP.NET Core
This article explains how to get started with WebSockets in ASP.NET Core. WebSocket (RFC 6455) is a protocol that enables two-way persistent ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found