Multiple http filters

[akhilsharma.in (Akhil Sharma)]

We've multiple conf files which have http filters in them. And they are running simultaneously at a time. Running multiple http filters at a time seems to give us problems. Please help us to understand if its the behavior of Logstash that it cannot run multiple http filter conf files at a time or we need to improve it?

TIA

[akhilsharma.in (Akhil Sharma)]

There are 3 conf files which are making http requests separately every 5 minutes. On seeing the logs, we have seen if any of these files run at a same time then only one call processes and other gives error.

We also see below error log at the end of log file, after this Logstash starts again automatically.

Have pasted the error log below:

[2020-04-02T06:01:00,576][WARN ][logstash.inputs.udp ] UDP listener died {:exception=>java.nio.channels.ClosedSelectorException, :backtrace=>["sun.nio.ch.SelectorImpl.keys(SelectorImpl.java:68)", "org.jruby.util.io.SelectorPool.put(SelectorPool.java:88)", "org.jruby.util.io.SelectExecutor.selectEnd(SelectExecutor.java:59)", "org.jruby.util.io.SelectExecutor.go(SelectExecutor.java:44)", "org.jruby.RubyIO.select(RubyIO.java:3405)", "org.jruby.RubyIO$INVOKER$s$0$3$select.call(RubyIO$INVOKER$s$0$3$select.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:743)", "org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)", "org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:161)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:109)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:95)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:145)", "amp.logstash.logstash_minus_6_dot_3_dot_0.logstash_minus_core.lib.logstash.pipeline.RUBY$method$inputworker$0(/amp/logstash/logstash-6.3.0/logstash-core/lib/logstash/pipeline.rb:512)", "amp.logstash.logstash_minus_6_dot_3_dot_0.logstash_minus_core.lib.logstash.pipeline.RUBY$method$inputworker$0$VARARGS(/amp/logstash/logstash-6.3.0/logstash-core/lib/logstash/pipeline.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:77)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:93)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:145)", "amp.logstash.logstash_minus_6_dot_3_dot_0.logstash_minus_core.lib.logstash.pipeline.RUBY$block$start_input$1(/amp/logstash/logstash-6.3.0/logstash-core/lib/logstash/pipeline.rb:505)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:145)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71)", "org.jruby.runtime.Block.call(Block.java:124)", "org.jruby.RubyProc.call(RubyProc.java:289)", "org.jruby.RubyProc.call(RubyProc.java:246)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104)", "java.lang.Thread.run(Thread.java:748)"]}
[2020-04-02T06:01:00,607][ERROR][org.logstash.Logstash ] java.lang.OutOfMemoryError: Java heap space