HOAB

History of a bug

Cannot create GC thread but a lot of memory

Rédigé par gorki Aucun commentaire

Problem :

Launching a JVM I have the message : "Cannot create GC thread. Out of system resources"

  • Enough memory
  • Enough swap
  • Enough ulimit
  • Enough threads-max
  • Enough CPU

Event extend the PID limit...

Important (at the end) : debian version = 10.11

Solution :

After a hours of googling, I found :

But none of these solutions works and none was matching the number I had :

  • number of open files < ulimit -n
  • maximum process/tasks < ulimit -u

But in a thread, I found something that was working : UserTasksMax.
I'm running SystemD, I have around 10805 task running for my user.
And from : https://manpages.debian.org/stretch/systemd/logind.conf.5.en.html

UserTasksMax=

Sets the maximum number of OS tasks each user may run concurrently. This controls the TasksMax= setting of the per-user slice unit, see systemd.resource-control(5) for details. If assigned the special value "infinity", no tasks limit is applied. Defaults to 33%, which equals 10813 with the kernel's defaults on the host, but might be smaller in OS containers.

For my suspect PID (a lot of files) :

  • cat /proc/21890/status | grep Thread => 1 thread
  • ls /proc/21890/task | wc
  • confirmed by the usual command : ps -eLf | grep calrisk | wc

I have around 10805 threads running for a given JVM very close to the limit.

Complete guide :

https://www.journaldufreenaute.fr/nombre-maximal-de-threads-par-processus-sous-linux/

Parameters not present in all man page, it could grown up to 12288 on latest version.

To be check !

 

 

Fil RSS des articles de ce mot clé