I am new for logstash 5.0 and tried to create custom filter at 'PATH/logstash/filters/global_config.rb' with the name of global config but I am getting error as:
bin/logstash -f conf_d/
Sending Logstash logs to /usr/local/seceon/ashish/logstash-5.0.0/logs which is now configured via log4j2.properties.
[2016-11-04T15:41:44,592][ERROR][logstash.agent ] fetched an invalid config {:config=>"input { stdin { } }\n\nfilter {\n grok {\n match => { "message" => "%{COMBINEDAPACHELOG}" }\n }\n date {\n match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]\n }\n global_config {\n }\n}\n\noutput {\n elasticsearch { hosts => ["localhost:9200"] }\n stdout { codec => rubydebug }\n}\n\n", :reason=>"Couldn't find any filter plugin named 'global_config'. Are you sure this is correct? Trying to load the global_config filter plugin resulted in this error: LoadError"}
I don't know what's I am doing wrong here can someone please help me ?
Same global-config.rb is working fine with logstash-2.4.0
You have an odd number of brackets in the error report. So check your config file and make sure you have properly placed the brackets correctly and didn't forget one . Also if you can post the config file it will help.
I have a very small config with input/filter/output .
This is the config file
[root@cce-logs-processor logstash-5.0.0]# cat conf_d/test.conf
input { stdin { } }
filter {
global_config {
}
}
output {
stdout { codec => rubydebug }
}
and this is the error
[2016-11-07T11:06:34,665][DEBUG][logstash.agent ] Reading config file {:config_file=>"/usr/local/seceon/ashish/logstash-5.0.0/conf_d/test.conf"}
it's here logstash/inputs/stdin
it's here stdin
it's here logstash/codecs/line
it's here line
[2016-11-07T11:06:34,690][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/@id = "line_b0144465-2a1e-42ac-a98d-6e521933dfda"
[2016-11-07T11:06:34,690][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/@enable_metric = true
[2016-11-07T11:06:34,690][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/@charset = "UTF-8"
[2016-11-07T11:06:34,690][DEBUG][logstash.codecs.line ] config LogStash::Codecs::Line/@delimiter = "\n"
[2016-11-07T11:06:34,692][DEBUG][logstash.inputs.stdin ] config LogStash::Inputs::Stdin/@id = "a2cbc585fb561768566d789e197abb1de72fe3cb-1"
[2016-11-07T11:06:34,692][DEBUG][logstash.inputs.stdin ] config LogStash::Inputs::Stdin/@enable_metric = true
[2016-11-07T11:06:34,692][DEBUG][logstash.inputs.stdin ] config LogStash::Inputs::Stdin/@codec = <LogStash::Codecs::Line id=>"line_b0144465-2a1e-42ac-a98d-6e521933dfda", enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">
[2016-11-07T11:06:34,692][DEBUG][logstash.inputs.stdin ] config LogStash::Inputs::Stdin/@add_field = {}
[2016-11-07T11:06:34,693][DEBUG][logstash.plugin ] Problems loading the plugin with {:type=>"filter", :name=>"global_config", :path=>"logstash/filters/global_config"}
[2016-11-07T11:06:34,699][ERROR][logstash.agent ] fetched an invalid config {:config=>"input { stdin { } }\n\nfilter {\n global_config {\n }\n}\n\noutput {\n stdout { codec => rubydebug }\n}\n\n", :reason=>"Couldn't find any filter plugin named 'global_config'. Are you sure this is correct? Trying to load the global_config filter plugin resulted in this error: LoadError", :backtrace=>["/usr/local/seceon/ashish/logstash-5.0.0/logstash-core/lib/logstash/plugin.rb:132:in lookup'", "/usr/local/seceon/ashish/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:413:inplugin'", "(eval):12:in initialize'", "org/jruby/RubyKernel.java:1079:ineval'", "/usr/local/seceon/ashish/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:90:in initialize'", "/usr/local/seceon/ashish/logstash-5.0.0/logstash-core/lib/logstash/agent.rb:195:increate_pipeline'", "/usr/local/seceon/ashish/logstash-5.0.0/logstash-core/lib/logstash/agent.rb:87:in register_pipeline'", "/usr/local/seceon/ashish/logstash-5.0.0/logstash-core/lib/logstash/runner.rb:258:inexecute'", "/usr/local/seceon/ashish/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/local/seceon/ashish/logstash-5.0.0/logstash-core/lib/logstash/runner.rb:174:inrun'", "/usr/local/seceon/ashish/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/local/seceon/ashish/logstash-5.0.0/lib/bootstrap/environment.rb:68:in(root)'"]}
I see the same issue with my custom output plugin.
Attempt to start logstash 5.0 with a custom output plugin (Ensured that I ve configured path.plugins correctly in the logstash.yml ) fails with the following error -
Trying to load the XXXXX output plugin resulted in this error: LoadError
I feel that for some reasons the register method in the plugin is not getting invoked. I debugged and see that the exception is being thrown at: registry.rb:69:in `lookup'
which means that the plugin registration indeed did not happen. And hence there is no entry for path,klass for my custom o/p plugin.
This was working all fine until logstash 2.2.1.
I browsed through the documentation for breaking changes, and followed whatever is being asked for. However, still I see this error.
Please note: I am not using plugin-install. I am trying to use the output plugin in a standalone fashion as a java based custom plugin.
Any help on this would be appreciated.
Thanks!
Yogesh
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
. Also if you can post the config file it will help.