PendingMessagesQueue growth is unbound

I am running MQTT server in a docker container, and i get strange behaviour where the amount of memory bloats into gigabytes.

The system is Producer -> Server -> MultipleConsumers The producer also uses MQTTnet, and sends ~100 messages per second, QOS 0 The consumers use MQTTnet too. they often disconnect without closing connection properly. The behaviour only occurs if the clients connect, and then disconnect. If the server is started, the producer is connected and producing messages but no client connect, then memory use stays very low (<100MB) It is not obvious to me why this is happening at this point. I could not reproduce it previously on my local machine in VS debugger, but I will try again soon.

Kubernetes Version - 1.7 .Net SDK / Runtime :

• 2.0
• 2.1 preview 2

There is a related issue - https://github.com/dotnet/coreclr/issues/14991#issuecomment-348428003 However I made sure that this isn’t the issue appearing here by running GC.Collect() every second - the issue still persists.

Server Code

public static readonly DateTime StartTimeUTC = DateTime.UtcNow;

static void Main(string[] args)
{
   Console.WriteLine("Server Started!");
   Task.Run(async () => await StartServer()).Wait();
}

private static async Task StartServer()
{
	var optionsBuilder = new MqttServerOptionsBuilder()
		.WithConnectionBacklog(100)
		.WithDefaultEndpointPort(1883);
	var mqttServer = new MqttFactory().CreateMqttServer();
	await mqttServer.StartAsync(optionsBuilder.Build());
	var ct = new CancellationTokenSource();
	var heartbeatTask = Task.Run(async () => await ServerHeartbeat(ct.Token, mqttServer)); 
  
	Console.WriteLine("Type 'quit' to exit");

	string input = string.Empty; 
	while (true)
	{
		//Docker has a habit of sending random shit into CMD
		char c = (char)Console.Read();
		input +=  c; 

	   if (input.Contains("quit")) 
		{
			Console.WriteLine("Shutting Down...");
			break;
		}
		else if (input.Length > 10)
		{
			input = input.Substring(input.Length - 3); 
		}
	}
	ct.Cancel();
	await heartbeatTask;
	await mqttServer.StopAsync();
}

private static async Task ServerHeartbeat(CancellationToken token, IMqttServer server)
{
  long heartbeat =0;
  while (token.IsCancellationRequested == false)
  {
	var message = new MqttApplicationMessageBuilder()
	 .WithTopic("$SYS/broker/uptime")
	 .WithPayload($"{DateTime.UtcNow - StartTimeUTC}")
	 .WithAtMostOnceQoS()
	 .WithRetainFlag(false)
	 .Build();
	await server.PublishAsync(message);

	message = new MqttApplicationMessageBuilder()
	 .WithTopic("$SYS/broker/time")
	 .WithPayload($"{DateTime.UtcNow}")
	 .WithAtMostOnceQoS()
	 .WithRetainFlag(false)
	 .Build();
	await server.PublishAsync(message);
	await Task.Delay(1000);
}

Am I doing something dumb? I will update the ticket as I continue testing.

[VLorz]

Hi Christian, same issue with version 2.7.5-rc1, yet it took longer than with previous version. Regards, Victor

[chkr1011]

Hi, thank you for sharing this issue and providing details. I will release a new RC soon with several perforrmance optimizations including some memory related optimizations. Best regards Christian