Hi everyone,
I have a hard time with logstash service. My server was set up to send filebeat to a kafka server and everything was fine until yesterday where I had the following error when debugging logstash:
` OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000ca660000, 899284992, 0) failed; error='Not enough space' (errno=12)
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 899284992 bytes for committing reserved memory.`
Right now the physical memory looks like this:
` total used free shared buff/cache available
Mem: 3944 3028 465 2 450 684
Swap: 0 0 0`
in my jvm.options, the heap space was originally set as:
-Xms1g -Xmx1g
I tried to lower the heap space (any value below 1G), but if I do this then logstash starts to work but it stops very quickly and I get another type of error. It seems that logstash can not work with a lower heap space. In particular this is the error I get when lowering the heap space:
`[ERROR] 2020-11-11 15:21:29.024 [Ruby-0-Thread-8: :1] javapipeline - Worker loop initialization error {:pipeline_id=>"main", :error=>"Missing Filter End Vertex", :exception=>Java::JavaLang::IllegalStateException, :stacktrace=>"org.logstash.config.ir.CompiledPipeline$CompiledExecution.lambda$compileFilters$1(org/logstash/config/ir/CompiledPipeline.java:386)\njava.util.Optional.orElseThrow(java/util/Optional.java:408)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.compileFilters(org/logstash/config/ir/CompiledPipeline.java:386)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.<init>(org/logstash/config/ir/CompiledPipeline.java:369)\norg.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.<init>(org/logstash/config/ir/CompiledPipeline.java:329)\norg.logstash.config.ir.CompiledPipeline.buildExecution(org/logstash/config/ir/CompiledPipeline.java:153)\norg.logstash.execution.WorkerLoop.<init>(org/logstash/execution/WorkerLoop.java:63)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(jdk/internal/reflect/NativeConstructorAccessorImpl.java:62)\njdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(jdk/internal/reflect/DelegatingConstructorAccessorImpl.java:45)\njava.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:490)\norg.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:253)\norg.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:939)\norg.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.init_worker_loop(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:499)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$init_worker_loop$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)\nusr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:267)\norg.jruby.RubyProc.call(org/jruby/RubyProc.java:318)\njava.lang.Thread.run(java/lang/Thread.java:834)", :thread=>"#<Thread:0x7d1929f7 sleep>"}`
Is there anoyone can help me to fix this issue?