Hi.
I am trying to create a new ELK stack using docker compose. My ES and Kibana containers are running fine.
However i am having issues with logstash. Originally it was starting using the docker compose file. But the container was crashing because it was using too much memory. So i tried to edit the docker compose file and add in a mem_limit and a "LS_JAVA_OPTS=-Xms1g -Xmx1g" setting. Once i did that the container wont start, i get the following error. (but i dont know 100% if this was due to me adding those or not. i have since removed them again but i still get the same error.)
[ERROR][logstash.javapipeline ][main] Worker loop initialization error {:pipeline_id=>"main", :error=>"Missing Filter End Vertex", :exception=>Java::JavaLang::IllegalStateException, :stack
trace=>"org.logstash.config.ir.CompiledPipeline$CompiledExecution.lambda$compileFilters$1(org/logstash/config/ir/CompiledPipeline.java:401)\njava.util.Optional.orElseThrow(java/util/Optional.java:403)\norg.logstash.c
onfig.ir.CompiledPipeline$CompiledExecution.compileFilters(org/logstash/config/ir/CompiledPipeline.java:401)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.<init>(org/logstash/config/ir/CompiledPipeline.j
ava:386)\norg.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.<init>(org/logstash/config/ir/CompiledPipeline.java:337)\norg.logstash.config.ir.CompiledPipeline.buildExecution(org/logstash/config/ir/Com
piledPipeline.java:156)\norg.logstash.execution.WorkerLoop.<init>(org/logstash/execution/WorkerLoop.java:65)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\njdk.internal.reflect.Nativ
eConstructorAccessorImpl.newInstance(jdk/internal/reflect/NativeConstructorAccessorImpl.java:77)\njdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(jdk/internal/reflect/DelegatingConstructorAccessorI
mpl.java:45)\njava.lang.reflect.Constructor.newInstanceWithCaller(java/lang/reflect/Constructor.java:499)\njava.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:480)\norg.jruby.javasupport.Java
Constructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:133)\norg.jruby.RubyClass.new(org/jruby/RubyClass.java:949)\norg.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newIn
stance.gen)\nRUBY.init_worker_loop(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:586)\nRUBY.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:289)\norg.jruby.RubyProc.cal
l(org/jruby/RubyProc.java:354)\njava.lang.Thread.run(java/lang/Thread.java:840)", :thread=>"#<Thread:0x36ef5a5d /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
the above error is duplicated twice. and then followed by this:
[INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2024-08-07T08:39:21,704][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2024-08-07T08:39:21,714][INFO ][logstash.runner ] Logstash shut down.
Just to note , i hadnt set any custom logstash config or pipelines. I am just trying to start the default logstash container using the default elastic image for the latest version.
the strange thing with this is that it was working with the docker compose file. i even tried removing the docker image and redownloading it.
my docker compose is just the default one on the elastic guide.
any ideas would be amazing?
thanks