Listening stops with many connections (pub/sub)

My demo bench is a small Net Core app with 20 connections as publishers, and other 20 as listeners. The broker is reached as 127.0.0.1 on Windows 10. Still I haven’t understand what’s going on, but I found an issue at this point: https://github.com/nats-io/csharp-nats/blob/d2ed82da04b2d2ddee770f1424302c0faf482289/NATS.Client/Conn.cs#L1047 Basically, the task does not start although apparently there’s no reason on that. The result is that the inner method is not called, and no message is listen to any of the subscribers. I solved turning the line as: new Thread(() => spinUpSocketWatchers()).Start(); Another way is to calling the method directly, but I guess you don’t want to halt the caller. I’ll dig into a little more, but…BAH!..let me know your opinion on.

[sebastien-curutchet]

If the threadpool doesn’t have enough available threads to do the connection, you have a timeout. It is because of this code client.ConnectAsync(s.url.Host, s.url.Port).Wait(TimeSpan.FromMilliseconds(timeoutMillis)). I have a similar issue on a stressed environment. I’m trying to implement the timeout by another way based on BeginConnect /EndConnect and TCPClient.Client.Poll method.

For now I did a workaround which avoids Timeout but doesn’t resolve the delay to execute the Connect method.

public static bool Connect(this TcpClient client, string host, int port, int actionTimeoutInMs, int totalTimeoutInMs = int.MaxValue)
        {
            client.Client.SendTimeout = totalTimeoutInMs;
            ManualResetEvent mre = new ManualResetEvent(false);
            var task = Task.Run(() =>
            {
                mre.Set();
                client.Connect(host, port);
            });
            if (mre.WaitOne(totalTimeoutInMs))
            {
                if (!task.Wait(actionTimeoutInMs))
                {
                    return false;
                }
                return true;
            }
            return false;
        }

[sixlettervariables]

You’re basically exhausting the thread pool. On my 32 core workstation, things got unwieldy past the 500 thread mark…

Any reason why you need so many NATS connections? Within an application one connection can handle arbitrarily many publishers and subscribers.

Your second example fails because most of the publishers finish publishing before the subscribers are even ready to listen.