Cant find installed plugin

Elastic Stack Logstash
1

Hi I am creating a new logstash output plugin and I'm trying to test it with logstash 5.5.1
I install it with bin/logstash-plugin install path/to/logstash-output-azure-0.1.1.gem
the out put is

Validating /path/to/logstash-output-azure-0.1.1.gem
Installing logstash-output-azure
Installation successful

my plugin has this

class LogStash::Outputs::LogstashAzureBlobOutput < LogStash::Outputs::Base
  
  config_name "logstash_output_azure"

  config :storage_account_name, valdiate: :string, required: false

  config :storage_access_key, valdiate: :string, required: false

  config :container_name, valdiate: :string, required: false

  config :size_file, validate: :number, default: 1024 * 1024 * 5
  config :time_file, validate: :number, default: 15
  config :restore, validate: :boolean, default: true
  config :temporary_directory, validate: :string, default: File.join(Dir.tmpdir, 'logstash')
  config :prefix, validate: :string, default: ''
  config :upload_queue_size, validate: :number, default: 2 * (Concurrent.processor_count * 0.25).ceil
  config :upload_workers_count, validate: :number, default: (Concurrent.processor_count * 0.5).ceil
  config :rotation_strategy, validate: %w[size_and_time size time], default: 'size_and_time'
  config :tags, :validate => :array, :default => []
  config :encoding, :validate => ["none", "gzip"], :default => "none"

  attr_accessor :storage_account_name, :storage_access_key,:container_name,
    :size_file,:time_file,:restore,:temporary_directory,:prefix,:upload_queue_size,
    :upload_workers_count,:rotation_strategy,:tags,:encoding

my config file config/logstasg.conf

output {
    logstash_output_azure {
        storage_account_name => "someaccount"
        container_name => "somename"
        storage_access_key => "somekey"
    
    }
}

when i start logstash with bin/logstash -f config/logstash.conf
I get the following errors

[2017-08-15T15:32:38,631][ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=>"output", :name=>"logstash_output_azure", :path=>"logstash/outputs/logstash_output_azure", :error_message=>"NameError", :error_class=>NameError, :error_backtrace=>["/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/plugins/registry.rb:226:in `namespace_lookup'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/plugins/registry.rb:162:in `legacy_lookup'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/plugins/registry.rb:138:in `lookup'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/plugins/registry.rb:180:in `lookup_pipeline_plugin'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/plugin.rb:140:in `lookup'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/pipeline.rb:100:in `plugin'", "(eval):8:in `initialize'", "org/jruby/RubyKernel.java:1079:in `eval'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/pipeline.rb:72:in `initialize'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/pipeline.rb:156:in `initialize'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/agent.rb:286:in `create_pipeline'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/agent.rb:95:in `register_pipeline'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/runner.rb:314:in `execute'", "/path/to/Downloads/logstash-5.5.1/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/path/to/Downloads/logstash-5.5.1/logstash-core/lib/logstash/runner.rb:209:in `run'", "/path/to/Downloads/logstash-5.5.1/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/path/to/Downloads/logstash-5.5.1/lib/bootstrap/environment.rb:71:in `(root)'"]}
[2017-08-15T15:32:38,649][ERROR][logstash.agent           ] Cannot create pipeline {:reason=>"Couldn't find any output plugin named 'logstash_output_azure'. Are you sure this is correct? Trying to load the logstash_output_azure output plugin resulted in this error: Problems loading the requested plugin named logstash_output_azure of type output. Error: NameError NameError"}

I have checked and the plugin apears to be installed when using bin/logstash-plugin list

I don't know why it fails to work

1 Like
2

Try this

bin/logstash-plugin list --verbose

This Will list installed plugins with version information

3

I can see my plugin is installed with the correct version logstash-output-azure (0.1.1)
I think the problem might be with the configuration file

4

I think the problem might be related to this
Problems loading the plugin with {:type=>"output", :name=>"logstash_output_azure"}
that's not hte plugin name that is the config_name specified in the gem.
does the gem name and the config name need to be the same?

5

Plugin loading works a bit like this:

output {
  logstash_output_azure { ... }
}
  1. This causes Logstash to ask the internal plugin registry if any output plugin exists with the name logstash_output_azure.
  2. At first, this will not be found, so Logstash will try to look for it.
  3. Logstash will try to load a file logstash/outputs/logstash_output_azure (https://github.com/elastic/logstash/blob/v5.5.0/logstash-core/lib/logstash/plugins/registry.rb?utf8=āœ“#L153-L156)
  4. require may throw a LoadError if the file is not found, and Logstash ignores this error in the hopes that perhaps this plugin is actually defined elsewhere already.
  5. Then it asks the LogStash::Outputs namespace for any class with the matching config_name (https://github.com/elastic/logstash/blob/v5.5.0/logstash-core/lib/logstash/plugins/registry.rb?utf8=āœ“#L217-L228)
  6. If no plug is found in the namespace check, we raise NameError.

My guess is that your plugin fails silently at step 4 where Logstash is expecting your plugin to be logstash/outputs/logstash_output_azure.rb but fails to load. You can check this in your logstash by editing logstash-core/lib/logstash/plugins/registry.rb in the legacy_lookup method and printing something in the rescue LoadError block.

7

Hello, Im working in the same plugin as @jaime_loom as a contributor. And I saw that there was a commit to print more specific error messages (https://github.com/elastic/logstash/pull/8147/files).

And now I can see that it didn't find the plugin on logstash/outputs/ indeed and threw the LogStash::PluginLoadingError. But the question is, why it didn't find? What are we missing ? Something on the conf file? Something with the gemspec? The plugin was validated and installed. should we manually put it somewhere else?

8

You can see more details in my post on https://github.com/elastic/logstash/issues/6834

9

nevermind, I found out what was wrong, it was some file name

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.