Client connection is lost with no obvious reason

I implemented an integration test that uses TcpClient and TcpServer. This test fails sporadically because the client loses the connection (e.g., disconnecting from 127.0.0.1:12345 due to connection lost) when trying to PollSocket, though the server is up and running.

I’ve treated this problem by increasing ConnectionLostEvaluationIntervalMs so that PollSocket is not called. Then the test is passed, and the client communicates correctly with the server.

The implementation of PollSocket does not provide an apparent reason for the connection loss:

private bool PollSocket()
{
    try
    {
        if (_client.Client == null || !_client.Client.Connected)
            return false;

        /* pear to the documentation on Poll:
         * When passing SelectMode.SelectRead as a parameter to the Poll method it will return 
         * -either- true if Socket.Listen(Int32) has been called and a connection is pending;
         * -or- true if data is available for reading; 
         * -or- true if the connection has been closed, reset, or terminated; 
         * otherwise, returns false
         */
        if (!_client.Client.Poll(0, SelectMode.SelectRead))
            return true;

        var buff = new byte[1];
        var clientSentData = _client.Client.Receive(buff, SocketFlags.Peek) != 0;
        return clientSentData; //False here though Poll() succeeded means we had a disconnect!
    }
    catch (SocketException)
    {
        return false;
    }
}

Thereby, it could be good to include some logging.

I’d like to know if I misunderstood some concepts or if there’re flaws in the PollSocket implementation?

[zllvm]

I’ve done the following change in PollSocket method:

catch (SocketException ex)
{
    Logger?.Invoke($"{Header}poll socket from {ServerIpPort} failed with ex = {ex}");
    return ex.SocketErrorCode == SocketError.TimedOut;
}

After that, I deployed it to the dev server and started a TCP connection that has been up and running for two days with no issues. I could test it by running various commands on the open connection, which worked perfectly. The log entry that I’ve added $"{Header}poll socket from {ServerIpPort} failed with ex = {ex}" appeared several time times (~20) in the monitoring system with Operation timed out exception.

Evidently, PollSocket introduces a harmful behavior by disconnecting an existing connection. I don’t know if it is possible to make it fully reliable. If not, then it should be allowed to override or disable it.

What is your opinion on this, @jchristn? Could we re-open this issue?

[jchristn]

Please re-open if this doesn’t address the issue. Cheers!