Logastash Exec plugin Cannot allocate memory

Elastic Stack Logstash
1

I am running Logstash 7.9.1 with 24 GB of maximum heap size configured in a Linux machine with 30 GB Memory and configured a pipeline to run a python program with Logstash exec plugin I see the below error message in logs.

Error while running command {:command=>"/usr/bin/python3 /opt/logstash/config/sql.py", :e=>#<Errno::ENOMEM: Cannot allocate memory - /usr/bin/python3>,

I did observe total JVM utilization is less than 15 GB. Below is my Logstash input configuration

input {
  exec {
    command => "/usr/bin/python3 /opt/logstash/config/sql.py"
    interval => 180

  }
}
2

If you use an exec plugin then the logstash JVM will fork. That creates a copy of the JVM's memory (excluding most shared memory segments). That means that on a 30GB server you will struggle to have the JVM memory be larger than 14 GB, so I would suggest a heap size of 12 GB at most. If you still get ENOMEM at 12 GB then keep reducing the heap size.

1 Like
3

@Badger Thanks for your quick reply issue got resolved after configuring the maximum JVM heap to 14 GB.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.