Logstash plugin installed but not found

Elastic Stack Logstash
1

I have installed a logstash plugin logstash-input-jms. I build the .gem file and used the bin/plugin install command. It shows it installed successfully, yet when i run bin/plugin list, it is not listed. I tried with both 1.5.0 and the latest 1.5.1

I have searched extensively but documentation here is limited for both this plugin and this issue.

2

I am facing the similar issue with the example which is given in the input plugin page.
(https://www.elastic.co/guide/en/logstash/current/_how_to_write_a_logstash_input_plugin.html)

The only renamed the plugin's name from example to example2, even in the gemspec etc. I am trying to figure how set the namespace, which is expected in line#131 of plugin.rb of logstash-1.5.1/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.1-java/lib/logstash

When I debug it I get the following error message:
bin/logstash --debug -e 'input { example2{} } output {stdout { codec => rubydebug }}'
Compiled pipeline code:
@inputs = []
@filters = []
@outputs = []
@periodic_flushers = []
@shutdown_flushers = []

      @input_example2_1 = plugin("input", "example2")

      @inputs << @input_example2_1

      @output_stdout_2 = plugin("output", "stdout", LogStash::Util.hash_merge_many({ "codec" => ("rubydebug") }))

      @outputs << @output_stdout_2

def filter_func(event)
events = [event]
@logger.debug? && @logger.debug("filter received", :event => event.to_hash)
events
end
def output_func(event)
@logger.debug? && @logger.debug("output received", :event => event.to_hash)
@output_stdout_2.handle(event)

end {:level=>:debug, :file=>"logstash/pipeline.rb", :line=>"28", :method=>"initialize"}
Plugin not defined in namespace, checking for plugin file {:type=>"input", :name=>"example2", :path=>"logstash/inputs/example2", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"133", :method=>"lookup"}
The error reported is:
Couldn't find any input plugin named 'example2'. Are you sure this is correct? Trying to load the example2 input plugin resulted in this error: no such file to load -- logstash/inputs/example2

4

try the following format:
|-- lib
| -- logstash
| -----inputs
| ------- example2.rb
|-- logstash-input-example2-0.1.0.gem
|-- logstash-input-example2.gemspec
|-- spec
|----- logstash
|--------- inputs
|------------- example2.rb

Fix the gemspec also.

5

I had the same issue. I created a plugin and installed it with 'bin/plugin install --no-verify .gem', but it wouldn't show up when doing 'plugin list'.
My problem was I did not include the .gemspec file in the Gem::Specification, s.files array.
This GitHub issue below tipped me to the problem.
Installing empty plugins #3444

6

ElGreco, I checked those and the format was correct except under |--spec i have:
|--spec
|----inputs
|------jms_spec.rb
Note no logstash subdir under spec, and .rb file has _spec instead of simply jms.rb

I am attempting to install a plugin from github, not one i wrote. Which file should i check for this in?

7

Similar to @whyapenny, I have installed a custom logstash output plugin. I build the .gem and specified the new plugin .gem path in the Gemfile and then tried bin/plugin install --no-verify. It shows it installed successfully, yet when i run bin/plugin list, it is not listed. I tried with both 1.5.0 and the latest 1.5.1. I have seen similar issues posted but the suggested solution is not working. When I run with debug on, it explicitly says development group plugins were not included. What do I have to do to get a development group plugin installed properly?

8

Found solution?
I'm having the same issue today. I'm using logstash-1.5.0 and used the latest log stash-filter-example from GitHub today.

9

I had a similar issue as well, where bin/plugin list did not show my plugin. But, when I start logstash with my output plugin included in the config, it works just fine.

10

Had similar problems this week where I was building 'empty' plugins. If it could be helpful, this is how I build plugins without messing up my machine:

Install docker if not done yet.

Build following Dockerfile (docker build -t sbeaupre/jruby .):

FROM jruby:1.7

RUN apt-get update
&& apt-get install -y git
&& gem install bundler
&& bundle install

Then write your own plugin or clone a plugin from github (Do not just download a zip from github! For some reason, some/most plugins have dependencies on git during the build), eg:

git clone -b v1.0.7 https://github.com/logstash-plugins/logstash-output-elasticsearch.git

In the root plugin folder, execute following command:

docker run -it --rm --name build-it -v "$PWD":/usr/src -w /usr/src sbeaupre/jruby bundle install && gem build logstash-output-elasticsearch.gemspec

And you get the gem file in the root folder of the plugin.

Main disadvantage is that the bundle install takes some time, but it is a flexible setup.

I am also using docker to test different ELK configurations.

hth,

Sven

11

I still have not had success with this. I have reverted back to logstash 1.4.2 and kibana 3.

12

@fh8510 I added '*.gemspec' to the s.files array:

s.files = Dir[ 'lib/logstash/outputs/.rb', 'spec/**/.rb', '*.gemspec' ]

but the plugin is still not listed in the output of bin/plugin list even though it said it installed successfully.

13

Hi all, I have installed successfully the plugin, I can see the plug in listed but when I try to use it I get the following:

The error reported is:
Couldn't find any filter plugin named 'rawstatistics_cms'. Are you sure this is correct? Trying to load the rawstatistics_cms filter plugin resulted in this error: no such file to load -- logstash/filters/rawstatistics_cms

1 Like
14

Yes. I have also experienced this issue, posted about it asking for help - even offered a bounty!

Nothing. Nadda. Zippo. Zilch.

The inability to develop, install and use my own plugins is destroying my commitment to Logstash. Before the new plugin manager it was a walk in the park :frowning:

15

Hi

Do we have any solution for this, i am facing this problem with file input plugin?

I am getting this error

Plugin not defined in namespace, checking for plugin file

Here is my config file, basically i am loading csv file in ES

input {  
      file {
	      sincedb_path => "E:/sincedbfile/sincedb_file1.txt"
          path => "E:/sincedbfile/abc.csv"
		  type=> "quality"
          start_position => "beginning"
      }
}


filter {  
    csv {
	    separator => ","
		columns => ["QualityEvent" , "EventNumber"]
    }
}


output {  
	stdout { codec => rubydebug }
    elasticsearch {
        action => "index"
        hosts => "127.0.0.1:9200"
        index => "abcde"
	

		
    }	
}
16

There is a variable in your plugin's main class with name - config_name.

Make sure that this is not set to empty string and has a name same as your plugin.

In case your plugin is logstash-input-xyz then
config_name "xyz"

Hope this helps :slight_smile: