Hi Team
I'm participating in the migration of a project from Logstash 6.4.3 to Logstash 7.10.2. The problem is that the new Logstash crashes at startup when one of our custom codecs is used in the configuration.
The error from the logstash-plain.log (formatted for readability):
Failed to execute action {
:action=>LogStash::PipelineAction::Create/pipeline_id:main,
:exception=>"Java::JavaLang::IllegalStateException",
:message=>"Unable to configure plugins: Missing Converter handling for full class name=org.jruby.gen.u20, simple name=RubyObject20",
:backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:119)",
"org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:83)",
"org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)",
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)",
"org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)",
"org.jruby.ir.instructions.InstanceSuperInstr.interpret(InstanceSuperInstr.java:84)",
"org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)",
"org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)",
"org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:86)",
"org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:73)",
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)",
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)",
"org.jruby.RubyClass.newInstance(RubyClass.java:939)",
"org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)",
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)",
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)",
"org.jruby.ir.instructions.CallBase.interpret(CallBase.java:549)",
"org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)",
"org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)",
"org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:92)",
"org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:191)",
"org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:178)",
"org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)",
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:396)",
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:205)",
"org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:325)",
"org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)",
"org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:116)",
"org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:137)",
"org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:60)",
"org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)",
"org.jruby.runtime.Block.call(Block.java:139)",
"org.jruby.RubyProc.call(RubyProc.java:318)",
"org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)",
"java.lang.Thread.run(Thread.java:748)"]
}
An exception happened when converging configuration {
:exception=>LogStash::Error,
:message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`",
:backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in `create'",
"org/logstash/execution/ConvergeResultExt.java:57:in `add'",
"/opt/logstash/logstash-core/lib/logstash/agent.rb:378:in `block in converge_state'"]
}
Here is a minimal Logstash config with which I was able to reproduce the issue:
input {
file {
path => [ "/var/log/logstash/chain-input.txt" ]
codec => "json"
}
}
output {
file {
path => "/var/log/logstash/chain-output.log"
codec => chain {
codec0 => json {}
}
}
}
As you can see the config uses "chain" codec which is developed internally in the company where I work. It was implemented 5 years ago and was working with previous Logstash versions (2..6) without problems.
How can I dig deeper into the issue? How to find out the line in the codec which causes the issue?
Or maybe it is a problem with Logstash or JRuby? How can I confirm it?