Hi team,
My logstash service is stopping again and again after few time, giving below error.
warning: thread "[main]>worker1" terminated with exception (report_on_exception is true):
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:413)
at org.logstash.common.LsQueueUtils.drain(LsQueueUtils.java:86)
at org.logstash.common.LsQueueUtils.drain(LsQueueUtils.java:56)
at org.logstash.ext.JrubyMemoryReadClientExt.readBatch(JrubyMemoryReadClientExt.java:61)
at org.logstash.execution.QueueReadClientBase.rubyReadBatch(QueueReadClientBase.java:106)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$worker_loop$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:333)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$worker_loop$0$VARARGS(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb)
at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)
at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)
at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$block$start_workers$2(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:304)
at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)
at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)
at org.jruby.runtime.Block.call(Block.java:124)
at org.jruby.RubyProc.call(RubyProc.java:295)
at org.jruby.RubyProc.call(RubyProc.java:274)
at org.jruby.RubyProc.call(RubyProc.java:270)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
at java.lang.Thread.run(Thread.java:748)
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:413)
at org.logstash.common.LsQueueUtils.drain(LsQueueUtils.java:86)
at org.logstash.common.LsQueueUtils.drain(LsQueueUtils.java:56)
at org.logstash.ext.JrubyMemoryReadClientExt.readBatch(JrubyMemoryReadClientExt.java:61)
at org.logstash.execution.QueueReadClientBase.rubyReadBatch(QueueReadClientBase.java:106)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$worker_loop$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:333)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$worker_loop$0$VARARGS(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb)
at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)
at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)
at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$block$start_workers$2(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:304)
at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)
at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)
at org.jruby.runtime.Block.call(Block.java:124)
at org.jruby.RubyProc.call(RubyProc.java:295)
at org.jruby.RubyProc.call(RubyProc.java:274)
at org.jruby.RubyProc.call(RubyProc.java:270)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
at java.lang.Thread.run(Thread.java:748)
warning: thread "[main]>worker6" terminated with exception (report_on_exception is true):
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:413)
at org.logstash.common.LsQueueUtils.drain(LsQueueUtils.java:86)
at org.logstash.common.LsQueueUtils.drain(LsQueueUtils.java:56)
at org.logstash.ext.JrubyMemoryReadClientExt.readBatch(JrubyMemoryReadClientExt.java:61)
at org.logstash.execution.QueueReadClientBase.rubyReadBatch(QueueReadClientBase.java:106)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$worker_loop$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:333)
at usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$block$start_workers$2(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:304)
at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)
at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)
at org.jruby.runtime.Block.call(Block.java:124)
at org.jruby.RubyProc.call(RubyProc.java:295)
at org.jruby.RubyProc.call(RubyProc.java:274)
at org.jruby.RubyProc.call(RubyProc.java:270)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
at java.lang.Thread.run(Thread.java:748)
[FATAL] 2019-10-09 06:11:38.110 [LogStash::Runner] runner - An unexpected error occurred! {:error=>#<ThreadError: interrupted in Thread::Queue#pop>, :backtrace=>["org/jruby/ext/thread/Queue.java:344:in pop'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/task.rb:38:inwait'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:379:in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:inrun'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:in run'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:inrun'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in `'"]}
[ERROR] 2019-10-09 06:11:38.200 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
Also it is not generating any log file. Please help me to get out of this shit.
Please update.
Thanks,
Suresh