I am using Logstash latest version 7.6.1 and I am using 4GB Ram in my machine.
When I trying to push data from Redshift to Elasticsearch using Logstash I am getting the Error:
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid30003.hprof ...
Heap dump file created [1187771647 bytes in 5.569 secs]
warning: thread "[main]>worker1" terminated with exception (report_on_exception is true):
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(java/util/Arrays.java:3664)
at java.lang.String.(java/lang/String.java:207)
at java.nio.HeapCharBuffer.toString(java/nio/HeapCharBuffer.java:567)
at java.nio.CharBuffer.toString(java/nio/CharBuffer.java:1241)
at org.jruby.RubyEncoding.decodeUTF8(org/jruby/RubyEncoding.java:297)
at org.jruby.RubyString.decodeString(org/jruby/RubyString.java:804)
at org.jruby.RubyString.toJava(org/jruby/RubyString.java:6247)
at org.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:901)
at org.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)
at java.lang.invoke.LambdaForm$DMH/696870185.invokeVirtual_L7_L(java/lang/invoke/LambdaForm$DMH)
at java.lang.invoke.LambdaForm$BMH/2022768790.reinvoke(java/lang/invoke/LambdaForm$BMH)
at java.lang.invoke.LambdaForm$MH/1016925085.delegate(java/lang/invoke/LambdaForm$MH)
at java.lang.invoke.LambdaForm$MH/1021082377.guard(java/lang/invoke/LambdaForm$MH)
at java.lang.invoke.LambdaForm$MH/1016925085.delegate(java/lang/invoke/LambdaForm$MH)
at java.lang.invoke.LambdaForm$MH/1021082377.guard(java/lang/invoke/LambdaForm$MH)
at java.lang.invoke.LambdaForm$MH/637524006.linkToCallSite(java/lang/invoke/LambdaForm$MH)
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_6_dot_4_minus_java.lib.manticore.client.request_from_options(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/client.rb:471)
at java.lang.invoke.LambdaForm$DMH/1100767002.invokeStatic_L9_L(java/lang/invoke/LambdaForm$DMH)
at java.lang.invoke.LambdaForm$BMH/1364958538.reinvoke(java/lang/invoke/LambdaForm$BMH)
at java.lang.invoke.LambdaForm$MH/789653861.delegate(java/lang/invoke/LambdaForm$MH)
at java.lang.invoke.LambdaForm$MH/228904713.guard(java/lang/invoke/LambdaForm$MH)
at java.lang.invoke.LambdaForm$MH/789653861.delegate(java/lang/invoke/LambdaForm$MH)
at java.lang.invoke.LambdaForm$MH/228904713.guard(java/lang/invoke/LambdaForm$MH)
[ERROR] 2020-04-06 06:29:06.417 [LogStash::Runner] Logstash - java.lang.OutOfMemoryError: Java heap space
I have increased the heap space using this command:
LS_HEAP_SIZE=3072m
But still, I am getting the same error.
But the record is 10K, then logstash is working fine. same time I trying to push one lakh record the logstash throw the error.
Kindly give me any suggestions.
Thanks and Regards.