Logstash JVM heap running out of space

(Exocomp) #1

Need some advice on how to resolve this issue. I setup an ELK box (1 box) that is working great, except every 4 days or so logstash runs out of JVM heap and then doesn't do much processing and I have to do a restart to fix it again.

See below screenshot of the JVM steadily increasing:

Any advice what I can do to resolve this problem, the ELK instance is setup with 4GB of RAM. I don't think it is a RAM capacity issue as it is fine initially and overtime fills up the heap.

(Aaron Mildenstein) #2

What version of Logstash? What does your configuration look like? Heap exhaustion is frequently the result of certain plugins being used in a particular configuration.

(Exocomp) #3


Version ---> 5.3.1
pipeline.workers ---> 6
pipeline.batch_delay ---> 5
pipeline.batch_size ---> 125

It is a single box with 4GB of RAM and 2 Cores and it is processing events / sec on average:

Input ---> 128
Filtered ---> 228
Output ---> 228

It consists of the following plugins:


  • beats
  • jdbc
  • http_poller
  • file


  • clone
  • ruby
  • mutate
  • json
  • grok
  • date
  • useragent
  • geoip


  • elasticsearch
  • graphite
  • statsd

(Aaron Mildenstein) #4

This is still rather incomplete without the actual configuration. My suspicions center on the ruby and clone filters, however. These can abuse memory.

(Exocomp) #5


I'll do some testing around clone and ruby filters to see if I can find a correlation and post my results. Thanks for the info.

(Exocomp) #6


Made some progress, well at least ruled out cloning as the problem. Enabled cloning but disabled some processing of the cloned events and it became stable testing for 4 days. See screenshot.

So I must be doing something in the processing of the clones only using the ruby filters, I will focus there and follow up.

(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.