Logstash and JAVA plugins

Elastic Stack Logstash
1

Wondering if someone would be able to give me a hand.

I'm trying to compile a java filter. I've followed the instructions here: https://github.com/logstash-plugins/logstash-filter-java_filter_example

but get this:

[2020-10-05T10:13:47,795][ERROR][logstash.plugins.registry] Tried to load a plugin's code, but failed. {:exception=>#<LoadError: no such file to load -- logstash/filters/logstash-filter-java_protocol>, :path=>"logstash/filters/logstash-filter-java_protocol", :type=>"filter", :name=>"logstash-filter-java_protocol"}
[2020-10-05T10:13:47,808][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (PluginLoadingError) Couldn't find any filter plugin named 'logstash-filter-java_protocol'. Are you sure this is correct? Trying to load the logstash-filter-java_protocol filter plugin resulted in this error: no such file to load -- logstash/filters/logstash-filter-java_protocol", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:119)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:82)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "media.david.data.ELK.logstash_minus_7_dot_9_dot_2.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/media/david/data/ELK/logstash-7.9.2/logstash-core/lib/logstash/java_pipeline.rb:44)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "media.david.data.ELK.logstash_minus_7_dot_9_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/media/david/data/ELK/logstash-7.9.2/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "media.david.data.ELK.logstash_minus_7_dot_9_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/media/david/data/ELK/logstash-7.9.2/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "media.david.data.ELK.logstash_minus_7_dot_9_dot_2.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/media/david/data/ELK/logstash-7.9.2/logstash-core/lib/logstash/agent.rb:357)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:748)"]}
warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):

but the plugin is listed?

./logstash-7.9.2/bin/logstash-plugin list --verbose

logstash-filter-java_protocol (1.0.0)

when I built it:

 ./gradlew gem

> Task :downloadJRuby
Download https://repo1.maven.org/maven2/org/jruby/jruby-dist/9.2.13.0/jruby-dist-9.2.13.0-bin.tar.gz

> Task :gem
WARNING:  open-ended dependency on jar-dependencies (>= 0) is not recommended
  use a bounded requirement, such as '~> x.y'
WARNING:  open-ended dependency on logstash-devutils (>= 0, development) is not recommended
  use a bounded requirement, such as '~> x.y'
WARNING:  See http://guides.rubygems.org/specification-reference/ for help
  Successfully built RubyGem
  Name: logstash-filter-java_protocol
  Version: 1.0.0
  File: logstash-filter-java_protocol-1.0.0.gem
2

I would expect a plugin named logstash-filter-java_protocol to show up in 'logstash-plugin list' as logstash-filter-logstash-filter-java_protocol

3

Thanks for your prompt response, appreciated..

This may be where i am confused (first time building a java plugin).

when i do run plugin-list, it is listed as what i built it as, and how it is used in the CONF file.

./logstash-7.9.2/bin/logstash-plugin list --verbose
logstash-filter-java_protocol (1.0.0)

CONF contains

logstash-filter-java_protocol{

this is the odd part, the error

Couldn't find any filter plugin named 'logstash-filter-java_protocol'

this is exactly what is listed in plugin list

logstash-filter-java_protocol

As an aside, i can remove it without issues.

./bin/logstash-plugin remove logstash-filter-java_protocol
Successfully removed logstash-filter-java_protocol

and install it:

./bin/logstash-plugin install --no-verify --local .... gem file
Installing logstash-filter-java_protocol
Installation successful

So it is odd it installs and removes, but cant be used? I'm sure it is something embarrassingly simple, but I'm at a loss..

4

Try changing that to

java_protocol {
5

and that was it.. thanks so much.. I've bashed my head trying to figure this out and it turns out it was something simple..

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.