Java heap space issue with logstash

Hi All,

I have been trying to ingest data which is of 6000 documents, its around 73 MB, but i'm repeatedly getting java heap space issue.

below is the error.

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid6498.hprof ...
Heap dump file created [4360270050 bytes in 6.815 secs]
warning: thread "[main]>worker3" terminated with exception (report_on_exception is true):
java.lang.OutOfMemoryError: Java heap space
        at org.jruby.RubyString.doClone(org/jruby/RubyString.java:1344)
        at org.logstash.Cloner.deep(org/logstash/Cloner.java:25)
        at org.logstash.Cloner.deepMap(org/logstash/Cloner.java:65)
        at org.logstash.Cloner.deep(org/logstash/Cloner.java:21)
        at org.logstash.Event.clone(org/logstash/Event.java:320)
        at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.clone(org/logstash/ext/JrubyEventExtLibrary.java:149)
        at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.clone(org/logstash/ext/JrubyEventExtLibrary.java:144)
        at java.lang.invoke.DirectMethodHandle$Holder.invokeSpecial(java/lang/invoke/DirectMethodHandle$Holder)
        at java.lang.invoke.LambdaForm$MH/0x00000008011cc440.invoke(java/lang/invoke/LambdaForm$MH)
        at java.lang.invoke.DelegatingMethodHandle$Holder.delegate(java/lang/invoke/DelegatingMethodHandle$Holder)
        at java.lang.invoke.LambdaForm$MH/0x00000008011c3040.guard(java/lang/invoke/LambdaForm$MH)
        at java.lang.invoke.DelegatingMethodHandle$Holder.delegate(java/lang/invoke/DelegatingMethodHandle$Holder)
        at java.lang.invoke.LambdaForm$MH/0x00000008011c3040.guard(java/lang/invoke/LambdaForm$MH)
        at java.lang.invoke.Invokers$Holder.linkToCallSite(java/lang/invoke/Invokers$Holder)
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_split_minus_3_dot_1_dot_8.lib.logstash.filters.split.filter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-split-3.1.8/lib/logstash/filters/split.rb:89)
        at java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java/lang/invoke/DirectMethodHandle$Holder)
        at java.lang.invoke.LambdaForm$MH/0x00000008011df440.invoke(java/lang/invoke/LambdaForm$MH)
        at java.lang.invoke.LambdaForm$MH/0x0000000800bd6840.invokeExact_MT(java/lang/invoke/LambdaForm$MH)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1814)
        at org.jruby.RubyArray$INVOKER$i$0$0$each.call(org/jruby/RubyArray$INVOKER$i$0$0$each.gen)
        at java.lang.invoke.LambdaForm$DMH/0x0000000801171840.invokeVirtual(java/lang/invoke/LambdaForm$DMH)
        at java.lang.invoke.LambdaForm$MH/0x00000008011e0040.invoke(java/lang/invoke/LambdaForm$MH)
        at java.lang.invoke.Invokers$Holder.linkToCallSite(java/lang/invoke/Invokers$Holder)
        at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_split_minus_3_dot_1_dot_8.lib.logstash.filters.split.filter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-split-3.1.8/lib/logstash/filters/split.rb:85)
[ERROR] 2020-08-02 13:04:51.142 [LogStash::Runner] Logstash - java.lang.OutOfMemoryError: Java heap space

I my config file below is the only filter i'm using

filter
       {
        json { source => "message" }
        split { field => "result" }
        prune { blacklist_names => [ "command", "host", "message" ] }
    }

Any advice on this please.

Thanks
Gautham

You could try moving the prune ahead of the split, but I doubt that will make a lot of difference.

Get a heap dump analyzer (like MAT) and see what the leak suspects in the dump are.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.