Getting class not found exception when invoking logstash http input with queue type as persisted

Hello All,

I am trying to setup ELK for internal event monitoring for our platform. Need help with the below,

Using logstash-7.10.2 with bundled jdk. with setting as per below snippets.

It is working fine when trying with queue.type as memory but when it is configured to persisted then I am getting the class not found error.

pipelines.yml

- pipeline.id: main
  pipeline.workers: 1
  pipeline.batch.size: 125
  queue.type: persisted
  path.config: config/sample/*.conf

sample.conf

 Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  http {
    id => "sample"
  }
}

output {
}

Error from stack trace when invoking logstash as below,

curl --header "x-api-id: 123" http://localhost:8080?data=dummy

Exception in thread "http-input-handler-executor[T#1]" java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/TSFBuilder
	at org.logstash.ObjectMappers.<clinit>(ObjectMappers.java:73)
	at org.logstash.Event.serialize(Event.java:465)
	at org.logstash.ackedqueue.Queue.write(Queue.java:363)
	at org.logstash.ackedqueue.ext.JRubyAckedQueueExt.rubyWrite(JRubyAckedQueueExt.java:125)
	at org.logstash.ext.JrubyAckedWriteClientExt.doPush(JrubyAckedWriteClientExt.java:78)
	at org.logstash.ext.JRubyWrappedWriteClientExt.push(JRubyWrappedWriteClientExt.java:102)
	at org.logstash.ext.JRubyWrappedWriteClientExt$INVOKER$i$1$0$push.call(JRubyWrappedWriteClientExt$INVOKER$i$1$0$push.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:833)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$push_decoded_event$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb:183)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$push_decoded_event$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$block$decode_body$1(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb:166)
	at org.jruby.ir.targets.YieldSite.yield(YieldSite.java:110)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$block$decode$2(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb:64)
	at org.jruby.ir.targets.YieldSite.yield(YieldSite.java:110)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_plain_minus_3_dot_0_dot_6.lib.logstash.codecs.plain.RUBY$method$decode$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-codec-plain-3.0.6/lib/logstash/codecs/plain.rb:35)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_plain_minus_3_dot_0_dot_6.lib.logstash.codecs.plain.RUBY$method$decode$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-codec-plain-3.0.6/lib/logstash/codecs/plain.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:197)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$block$decode$1(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb:62)
	at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
	at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
	at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:32)
	at org.jruby.runtime.Block.call(Block.java:147)
	at org.logstash.instrument.metrics.MetricExt.doTime(MetricExt.java:160)
	at org.logstash.instrument.metrics.AbstractSimpleMetricExt.time(AbstractSimpleMetricExt.java:65)
	at org.logstash.instrument.metrics.NamespacedMetricExt.doTime(NamespacedMetricExt.java:107)
	at org.logstash.instrument.metrics.AbstractNamespacedMetricExt.time(AbstractNamespacedMetricExt.java:64)
	at org.logstash.instrument.metrics.AbstractNamespacedMetricExt$INVOKER$i$1$0$time.call(AbstractNamespacedMetricExt$INVOKER$i$1$0$time.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneBlock.call(JavaMethod.java:656)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:197)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$method$decode$0(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb:61)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$method$decode$0$__VARARGS__(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:197)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$decode_body$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb:166)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$decode_body$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.message_handler.RUBY$method$onNewMessage$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http/message_handler.rb:30)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.message_handler.RUBY$method$onNewMessage$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http/message_handler.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.gen.LogStash$$Inputs$$Http$$MessageHandler_1195421663.onNewMessage(org/jruby/gen/LogStash$$Inputs$$Http$$MessageHandler_1195421663.gen:13)
	at org.logstash.plugins.inputs.http.MessageProcessor.processMessage(MessageProcessor.java:77)
	at org.logstash.plugins.inputs.http.MessageProcessor.run(MessageProcessor.java:62)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.TSFBuilder
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 55 more
Exception in thread "http-input-handler-executor[T#2]" java.lang.NoClassDefFoundError: Could not initialize class org.logstash.ObjectMappers
	at org.logstash.Event.serialize(Event.java:465)
	at org.logstash.ackedqueue.Queue.write(Queue.java:363)
	at org.logstash.ackedqueue.ext.JRubyAckedQueueExt.rubyWrite(JRubyAckedQueueExt.java:125)
	at org.logstash.ext.JrubyAckedWriteClientExt.doPush(JrubyAckedWriteClientExt.java:78)
	at org.logstash.ext.JRubyWrappedWriteClientExt.push(JRubyWrappedWriteClientExt.java:102)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$push_decoded_event$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb:183)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$push_decoded_event$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$block$decode_body$1(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb:166)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$block$decode$2(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb:64)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_plain_minus_3_dot_0_dot_6.lib.logstash.codecs.plain.RUBY$method$decode$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-codec-plain-3.0.6/lib/logstash/codecs/plain.rb:35)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_codec_minus_plain_minus_3_dot_0_dot_6.lib.logstash.codecs.plain.RUBY$method$decode$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-codec-plain-3.0.6/lib/logstash/codecs/plain.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:197)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$block$decode$1(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb:62)
	at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
	at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
	at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:32)
	at org.jruby.runtime.Block.call(Block.java:147)
	at org.logstash.instrument.metrics.MetricExt.doTime(MetricExt.java:160)
	at org.logstash.instrument.metrics.AbstractSimpleMetricExt.time(AbstractSimpleMetricExt.java:65)
	at org.logstash.instrument.metrics.NamespacedMetricExt.doTime(NamespacedMetricExt.java:107)
	at org.logstash.instrument.metrics.AbstractNamespacedMetricExt.time(AbstractNamespacedMetricExt.java:64)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$method$decode$0(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb:61)
	at workspace.logstash_minus_7_dot_10_dot_2.logstash_minus_core.lib.logstash.codecs.delegator.RUBY$method$decode$0$__VARARGS__(/workspace/logstash-7.10.2/logstash-core/lib/logstash/codecs/delegator.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:197)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$decode_body$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb:166)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.RUBY$method$decode_body$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.message_handler.RUBY$method$onNewMessage$0(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http/message_handler.rb:30)
	at workspace.logstash_minus_7_dot_10_dot_2.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_http_minus_3_dot_3_dot_6_minus_java.lib.logstash.inputs.http.message_handler.RUBY$method$onNewMessage$0$__VARARGS__(/workspace/logstash-7.10.2/vendor/bundle/jruby/2.5.0/gems/logstash-input-http-3.3.6-java/lib/logstash/inputs/http/message_handler.rb)
	at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
	at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
	at org.jruby.gen.LogStash$$Inputs$$Http$$MessageHandler_1195421663.onNewMessage(org/jruby/gen/LogStash$$Inputs$$Http$$MessageHandler_1195421663.gen:13)
	at org.logstash.plugins.inputs.http.MessageProcessor.processMessage(MessageProcessor.java:77)
	at org.logstash.plugins.inputs.http.MessageProcessor.run(MessageProcessor.java:62)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

Thanks,
Sivakumar

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.