Unable to run alongside Java VM

Describe the bug I am unable to start the Java VM alongside the Audio Bot. This is true for

• starting any Java program before the bot -> Bot won’t start, see logs
• starting any Java program when the bot is running -> Java won’t start, see logs

I’m unsure whether this is a bug/issue with your bot or some setup thing as dotnet and java don’t like each other. Not being the most advanced Linux user, I tried everything I could find about checking thread- and memory-limits, but without finding anything that’s wrong. Maybe someone on here can reproduce my problem or knows, what exactly could cause this.

To Reproduce

1. Start the bot
2. Start Java VM, i.e. with java -version -> crash

OR

1. Start any Java program (I originally tested with a “bigger” program, but the issue is also present with a program that just prints something in an endless loop and with limited memory for the VM and single threadstacks).
2. Start the bot -> crash

Version Version: 0.10.1+7 Branch: master CommitHash: 65aa9a18ca5f72f0be9adc6aa93571b9291a9398

Platform

• Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0 x86_64)
• VServer with 8GB RAM and 4 VCores
• dotnet --info:
  • Host (useful for support): Version: 3.1.1 Commit: a1388f194c
  • .NET Core runtimes installed: Microsoft.NETCore.App 2.2.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App] Microsoft.NETCore.App 3.1.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Log Starting TS3AudioBot after any Java program

2020-01-24 19:46:08.5672| INFO||Setup.LogHeader [============ TS3AudioBot started =============]
2020-01-24 19:46:08.6953| INFO||Setup.LogHeader [ Date/Time: Freitag, 24. Januar 2020 19:46:08
2020-01-24 19:46:08.7085| INFO||Setup.LogHeader [ Version: 0.10.1+7/master/65aa9a18
2020-01-24 19:46:08.7103| INFO||Setup.LogHeader [ Platform:   (64bit)
2020-01-24 19:46:08.7115| INFO||Setup.LogHeader [ Runtime: .NET Core (2.2.8) ServerGC:True GC:SustainedLowLatency
2020-01-24 19:46:08.7115| INFO||Setup.LogHeader [ Opus: libopus 1.1.2 (x64)
2020-01-24 19:46:08.7124| INFO||Setup.LogHeader [==============================================]
2020-01-24 19:46:09.6812|FATAL||Core.ExceptionHandler Critical program failure!
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Threading.Thread.StartInternal()
at TS3AudioBot.Helper.TickPool.AddWorker(TickWorker worker) in C:\projects\ts3audiobot\TS3AudioBot\Helper\TickPool.cs:line 53
at TS3AudioBot.Helper.TickPool.RegisterTick(Action method, TimeSpan interval, Boolean active) in C:\projects\ts3audiobot\TS3AudioBot\Helper\TickPool.cs:line 41
at TS3AudioBot.Helper.Environment.SystemMonitor.StartTimedSnapshots() in C:\projects\ts3audiobot\TS3AudioBot\Helper\Environment\SystemMonitor.cs:line 27
at TS3AudioBot.Core.Run(Boolean interactive) in C:\projects\ts3audiobot\TS3AudioBot\Core.cs:line 121
at TS3AudioBot.Core.Main(String[] args) in C:\projects\ts3audiobot\TS3AudioBot\Core.cs:line 55
2020-01-24 19:46:09.7232| INFO||Core.Dispose TS3AudioBot shutting down.
2020-01-24 19:46:09.7345| INFO||WebServer.Dispose WebServer is closing

Starting any Java program after TS3AudioBot

[0.084s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.lang.Thread.start0(java.base/Native Method)
at java.lang.Thread.start(java.base/Thread.java:803)
at java.lang.ref.Reference.<clinit>(java.base/Reference.java:303)

[Thomseeen]

So I tried the --no-llgc option and that runtime option without any positive results.

To check for the thread/task limitation is just opened a bunch of ssh-connections, after about 11 I wasn’t able to spawn any more and was unable to start the JVM with the same error as listed above… I had to close half of the ssh-connections to be able to start java -version… so I guess there is some weird task or thread limitation in place, which I can’t find. Guess I will try to contact my hoster’s support.

[Thomseeen]

Solved; it was a problem with the threads being limited by Ubuntu.

Problem:

My hoster uses Virtuozzo which limits the maximum processes per VM-instance in /proc/user_beancounters - numproc. This value is 400 for my type of server, which doesn’t explain the problems I get at around 60 tasks. It appears Ubuntu takes this value and sets the maximum allowed tasks, according to systemctl show --property=DefaultTasksMax, to 15% of that by default. 400*0.15=60 so that’s where the 60 tasks-limit comes from.

Solution:

DefaultTasksMax can be changed in /etc/systemd/system.conf (120 for me now). Everything works as expected now.