I have Elasticsearch and Kibana on a different host. I'm trying to get my Logstash instance to output to Elasticsearch on a remote host. I can forward my logs to Filebeat and then to Elastic, but I am trying to write/test a grok pattern for Mimecast logs.
I've been banging my head against the wall for a couple days. I've validated that my pipeline is good.
I'm guessing this is a permissions issue? Any ideas?
Here is my Mimecast.conf
input {
file {
path => "/tmp/mimecast/*.log"
}
}
filter{
grok {
match => { "message" => "%{TIMESTAMP_ISO8601:date}%{GREEDYDATA:Message}"}
}
}
output {
elasticsearch {
hosts => ["172.16.200.50:9200"]
index => "mimecast_index_%{+YYYYMMdd}"
}
}
And the error message
[2021-11-19T16:32:42,879][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2021-11-19T16:32:43,584][INFO ][org.reflections.Reflections] Reflections took 84 ms to scan 1 urls, producing 120 keys and 417 values
[2021-11-19T16:32:44,878][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//172.16.200.50:9200"]}
[2021-11-19T16:32:45,265][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://172.16.200.50:9200/]}}
[2021-11-19T16:32:45,407][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://172.16.200.50:9200/"}
[2021-11-19T16:32:45,453][ERROR][logstash.javapipeline ][main] Pipeline error {:pipeline_id=>"main", :exception=>#<NoMethodError: undefined method `[]' for nil:NilClass>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:426:in `get_es_version'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:243:in `block in healthcheck!'", "org/jruby/RubyHash.java:1415:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:237:in `healthcheck!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:328:in `update_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:86:in `update_initial_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:80:in `start'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:344:in `build_pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:62:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:106:in `create_http_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb:102:in `build'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:34:in `build_client'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.5-java/lib/logstash/outputs/elasticsearch.rb:275:in `register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:131:in `register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:68:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1820:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:585:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:240:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/etc/logstash/conf.d/mimecast.conf"], :thread=>"#<Thread:0x55b8c45e run>"}
[2021-11-19T16:32:45,457][INFO ][logstash.javapipeline ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2021-11-19T16:32:45,486][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2021-11-19T16:32:45,621][INFO ][logstash.runner ] Logstash shut down.
[2021-11-19T16:32:45,642][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.19.0.jar:?]
at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.19.0.jar:?]
at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:94) ~[?:?]