# Pipeline Worker Loop Initialization Error **URL:** \ **Category:** Logstash\ **Created:** [July 21, 2023, 10:55pm UTC](https://discuss.elastic.co/t/pipeline-worker-loop-initialization-error/338972 "2023-07-21T22:55:03Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![dro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dro/32/131811_2.png) [@dro](https://discuss.elastic.co/u/dro)\ **Post date:** [July 21, 2023, 10:55pm UTC](https://discuss.elastic.co/t/pipeline-worker-loop-initialization-error/338972/1 "2023-07-21T22:55:03Z")
Been running into an issue with Logstash failing to initialize due to pipeline worker error. Containerized version being used: `Logstash 8.8.2` Error: ```auto Using bundled JDK: /usr/share/logstash/jdk Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties [2023-07-21T22:44:15,293][INFO][logstash.runner] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties [2023-07-21T22:44:15,298][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"8.8.2", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.7+7 on 17.0.7+7 +indy +jit [x86_64-linux]"} [2023-07-21T22:44:15,301][INFO][logstash.runner] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dls.cgroup.cpuacct.path.override=/, -Dls.cgroup.cpu.path.override=/, -Xmx1g, -Xms1g, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED] [2023-07-21T22:44:15,311][INFO][logstash.settings] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"} [2023-07-21T22:44:15,315][INFO][logstash.settings] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"} [2023-07-21T22:44:15,680][INFO][logstash.agent] No persistent UUID file found. Generating new UUID {:uuid=>"ebc73fba-c1fa-4877-abd5-0eeb8524da29", :path=>"/usr/share/logstash/data/uuid"} [2023-07-21T22:44:16,689][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} [2023-07-21T22:44:16,890][INFO][org.reflections.Reflections] Reflections took 210 ms to scan 1 urls, producing 132 keys and 464 values [2023-07-21T22:44:16,974][INFO][logstash.javapipeline] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise. [2023-07-21T22:44:16,992][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#"} [2023-07-21T22:44:16,998][ERROR][logstash.javapipeline][main] Worker loop initialization error {:pipeline_id=>"main", :error=>"Missing Filter End Vertex", :exception=>Java::JavaLang::IllegalStateException, :stacktrace=>"org.logstash.config.ir.CompiledPipeline$CompiledExecution.lambda$compileFilters$1(org/logstash/config/ir/CompiledPipeline.java:401)\njava.util.Optional.orElseThrow(java/util/Optional.java:403)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.compileFilters(org/logstash/config/ir/CompiledPipeline.java:401)\norg.logstash.config.ir.CompiledPipeline$CompiledExecution.(org/logstash/config/ir/CompiledPipeline.java:386)\norg.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.(org/logstash/config/ir/CompiledPipeline.java:337)\norg.logstash.config.ir.CompiledPipeline.buildExecution(org/logstash/config/ir/CompiledPipeline.java:156)\norg.logstash.execution.WorkerLoop.(org/logstash/execution/WorkerLoop.java:65)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\njdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(jdk/internal/reflect/NativeConstructorAccessorImpl.java:77)\njdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(jdk/internal/reflect/DelegatingConstructorAccessorImpl.java:45)\njava.lang.reflect.Constructor.newInstanceWithCaller(java/lang/reflect/Constructor.java:499)\njava.lang.reflect.Constructor.newInstance(java/lang/reflect/Constructor.java:480)\norg.jruby.javasupport.JavaConstructor.newInstanceDirect(org/jruby/javasupport/JavaConstructor.java:237)\norg.jruby.RubyClass.new(org/jruby/RubyClass.java:911)\norg.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)\nRUBY.init_worker_loop(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:583)\nRUBY.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:289)\norg.jruby.RubyProc.call(org/jruby/RubyProc.java:309)\njava.lang.Thread.run(java/lang/Thread.java:833)", :thread=>"#"} [2023-07-21T22:44:17,001][ERROR][logstash.javapipeline][main] Pipeline error {:pipeline_id=>"main", :exception=>#, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:293:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:146:in `block in start'"], "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#"} [2023-07-21T22:44:17,001][INFO][logstash.javapipeline][main] Pipeline terminated {"pipeline.id"=>"main"} [2023-07-21T22:44:17,008][ERROR][logstash.agent] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create
, action_result: false", :backtrace=>nil} [2023-07-21T22:44:17,074][INFO][logstash.runner] Logstash shut down. [2023-07-21T22:44:17,079][FATAL][org.logstash.Logstash] Logstash stopped processing because of an error: (SystemExit) exit ``` Relevant Helm Chart section: ```auto logstash: enabled: true logstashPipeline: logstash.conf: |- {{ .Files.Get "logstash-pipelines/my_apps.conf" }} logstashConfig: logstash.yml: | http.host: 0.0.0.0 monitoring.elasticsearch.hosts: http://elasticsearch-master.elk.svc:9200 pipelines.yml: | - pipeline.id: myapps path.config: "/usr/share/logstash/pipeline/logstash.conf" extraEnvs: - name: "ELASTICSEARCH_USERNAME" valueFrom: secretKeyRef: name: elasticsearch-master-credentials key: username - name: "ELASTICSEARCH_PASSWORD" valueFrom: secretKeyRef: name: elasticsearch-master-credentials key: password secretMounts: - name: elasticsearch-master-certs secretName: elasticsearch-master-certs path: /usr/share/logstash/config/certs service: annotations: {} type: ClusterIP loadBalancerIP: "" ports: - name: beats port: 5045 protocol: TCP targetPort: 5045 - name: http port: 8080 protocol: TCP targetPort: 8080 ``` The pipeline is very simple to just test functionality: ```auto input { beats { port => 5044 codec => json { target => "[message]" } } } output { elasticsearch { hosts => ["elasticsearch-master.elk.svc:9200"] user => '${ELASTICSEARCH_USERNAME}' password => '${ELASTICSEARCH_PASSWORD}' manage_template => false ssl => true cacert => '/usr/share/logstash/config/certs/ca.crt' index => "logstash-%{[fields][index_name]}-%{+yyyy.MM.dd}" } } ``` The pipeline seems very simple and not sure why I am getting `error=>"Missing Filter End Vertex"`? I've read the docs to ensure that the pipeline overwrites the default `logstash.conf` which is what I did in the helm chart. Any ideas in how to troubleshoot this issue? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [July 22, 2023, 12:53am UTC](https://discuss.elastic.co/t/pipeline-worker-loop-initialization-error/338972/2 "2023-07-22T00:53:07Z")
> [@dro](#): > > The pipeline seems very simple and not sure why I am getting `error=>"Missing Filter End Vertex"`? This error normally happens when the pipeline is missing the `input` block. I would double check your `my_apps.conf` file and check if it is really replacing the `/usr/share/logstash/pipeline/logstash.conf` file in your pod. ---
**Author:** ![dro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dro/32/131811_2.png) [@dro](https://discuss.elastic.co/u/dro)\ **Post date:** [July 22, 2023, 5:15am UTC](https://discuss.elastic.co/t/pipeline-worker-loop-initialization-error/338972/3 "2023-07-22T05:15:29Z")
After your comment I decided to take a little break and came back. Indeed you are correct. I had a typo in my pathing for `.Files.Get` which was causing the pipeline to not get copied over and who knows what was overriding the default `logstash.conf` which resulted in the error. Thanks for the help! Please close topic. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 19, 2023, 5:16am UTC](https://discuss.elastic.co/t/pipeline-worker-loop-initialization-error/338972/4 "2023-08-19T05:16:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.