Waiting for input plugin, Dropping events to unblock input plugin, Pipeline terminated

[AdxDaz]

Hi Elastic team,

I was working in the next post "Locate json field with jsonpath (logstash) - #2 by Badger" and despite the code works, i have the next error when the position of the json value change:

[2022-11-15T09:30:17,191][WARN ][logstash.javapipeline    ][log_trans] Waiting for input plugin to close {:pipeline_id=>"log_trans", :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:18,192][WARN ][logstash.javapipeline    ][log_trans] Waiting for input plugin to close {:pipeline_id=>"log_trans", :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:19,202][ERROR][logstash.javapipeline    ][log_trans] Dropping events to unblock input plugin {:pipeline_id=>"log_trans", :count=>125, :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:19,304][ERROR][logstash.javapipeline    ][log_trans] Dropping events to unblock input plugin {:pipeline_id=>"log_trans", :count=>1, :thread=>"#<Thread:0x7371c607@/home/logstash/logstash-7.12.0/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-11-15T09:30:27,750][INFO ][logstash.javapipeline    ][log_trans] Pipeline terminated {"pipeline.id"=>"log_trans"}

This error kill the pipeline and i have to restart all logstash because don't start the pipeline on its own.

[AdxDaz]

Hi Every one,

I replied the error in my personal enviroment, and i could extract more information about the error.

[2022-11-17T14:10:06,662][ERROR][logstash.javapipeline    ][main] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"main", :error=>"", :exception=>Java::JavaLang::NullPointerException, :backtrace=>["org.logstash.config.ir.compiler.EventCondition$Compiler$UnexpectedTypeException.getUnexpectedTypeDetails(EventCondition.java:693)", "org.logstash.config.ir.compiler.EventCondition$Compiler$UnexpectedTypeException.<init>(EventCondition.java:681)", "org.logstash.config.ir.compiler.EventCondition$Compiler.compare(EventCondition.java:453)", "org.logstash.config.ir.compiler.EventCondition$Compiler.lambda$compareFieldToConstant$11(EventCondition.java:444)", "org.logstash.config.ir.compiler.Utils.filterEvents(Utils.java:47)", "org.logstash.generated.CompiledDataset3.compute(Unknown Source)", "org.logstash.generated.CompiledDataset2.compute(Unknown Source)", "org.logstash.generated.CompiledDataset2.compute(Unknown Source)", "org.logstash.generated.CompiledDataset2.compute(Unknown Source)", "org.logstash.generated.CompiledDataset4.compute(Unknown Source)", "org.logstash.generated.CompiledDataset2.compute(Unknown Source)", "org.logstash.generated.CompiledDataset2.compute(Unknown Source)", "org.logstash.generated.CompiledDataset2.compute(Unknown Source)", "org.logstash.generated.CompiledDataset5.compute(Unknown Source)", "org.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.compute(CompiledPipeline.java:329)", "org.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.compute(CompiledPipeline.java:323)", "org.logstash.execution.WorkerLoop.run(WorkerLoop.java:83)", "jdk.internal.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)", "java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)", "java.base/java.lang.reflect.Method.invoke(Method.java:566)", "org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:441)", "org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:305)", "org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:32)", "mnt.c.adx.Proyectos.logstash.logstash_minus_7_dot_12_dot_0.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_workers$5(/mnt/c/adx/Proyectos/logstash/logstash-7.12.0/logstash-core/lib/logstash/java_pipeline.rb:295)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:834)"], :thread=>"#<Thread:0x3a863873 sleep>"}
[2022-11-17T14:10:06,668][INFO ][filewatch.observingtail  ][main] QUIT - closing all files and shutting down.
[2022-11-17T14:10:06,670][WARN ][logstash.javapipeline    ][main] Waiting for input plugin to close {:pipeline_id=>"main", :thread=>"#<Thread:0x3a863873 run>"}
[2022-11-17T14:10:07,815][INFO ][logstash.javapipeline    ][main] Pipeline terminated {"pipeline.id"=>"main"}

I hope that this help.

Thanks

[Badger]

That looks a lot like this.

[AdxDaz]

Thanks @Badger, yes i should be very carefully when i apply the if condition, if the value doesn't match and logstash don't find in the excecution order, happens this exception.