Request threading issues?

Hi Colin, I’m doing some fairly basic tests at the moment to explore throughput. I’ve a system that I want to Request about 20 sources and aggregate the results.

If I just use publish (using the same approach as below), then I can comfortably publish 10,000 messages in <300ms on my PC. So I put together the following snippet with Request, and I was expecting something similar, but am seeing NATSTimeoutException instead.

                Msg msg = null;
                ConnectionFactory cf = new ConnectionFactory();
                var sw = new Stopwatch();
                sw.Start();
                using (var c = cf.CreateConnection())
                {
                    var tasks = Enumerable.Range(0, 10000).Select(x =>
                         Task.Factory.StartNew(() =>
                         {
                           msg = c.Request("AppointmentSearch", new AppointmentSearch().ToMsg(), 10 * 1000);
                         })
                    );
                    Task.WaitAll(tasks.ToArray(), TimeSpan.FromSeconds(10));
                }
                return sw.ElapsedMilliseconds;

Looking at gnatsd -V I can see a throughput of about 1 message per second. This doesn’t seem right to me? Is there anything I can look at?

[ColinSullivan1]

@chrismcv I agree, but think this has more to do with the task scheduler itself, than NATS code. I wrote a few tests using Tasks that failed quickly (running on VM), similar to your results. The tasks weren’t being scheduled in a performant manner, likely due to resources constraints. However, when I moved down to using threads I found the performance I was expecting. 300 threads requesting, with the responder randomly delaying up to 5 seconds, and the tests passed about the same time as the delay, indicating there was little to no contention in request calls. The responses were received were out of order, as expected, also indicating little contention. Each subscription does have its own channel, so they shouldn’t step on each other. This is an interesting problem… That being said, this needs to be addressed and I will continue looking.

[chrismcv]

I created the following method

 private Task<Msg> RequestAsync(IConnection c, string subject, byte[] data, int timeout)
        {
            var tcs = new TaskCompletionSource<Msg>();
            var ct = new CancellationTokenSource(timeout);
            ct.Token.Register(() => tcs.TrySetCanceled(), useSynchronizationContext: false);

            string inbox = c.NewInbox();

            var s = c.SubscribeAsync(inbox);
            s.AutoUnsubscribe(1);
            s.MessageHandler += (e, h) =>
            {
                tcs.TrySetResult(h.Message);
                try
                {
                    s.Unsubscribe();
                }
                catch { }
            };
            s.Start();
            c.Publish(subject, inbox, data);
            return tcs.Task;
        }

and using

Msg msg = null;
                var res = new ConcurrentBag<AppointmentSearchResponse>();
                ConnectionFactory cf = new ConnectionFactory();
                var sw = new Stopwatch();
                sw.Start();
                using (var c = cf.CreateConnection())
                {
                    var tasks = Enumerable.Range(0, 10000).Select(async x =>
                    {
                        msg = await RequestAsync(c, "AppointmentSearch", new AppointmentSearch().ToMsg(), 10 * 1000);
                        res.Add(msg.As<AppointmentSearchResponse>());
                    });
                    Task.WaitAll(tasks.ToArray(), 100000);
                }

I get much better throughput - e.g. 600ms for 10,000. I suspect the SubscribeSync in request is blocking other Tasks but SubscribeAsync isn’t.