[FATAL][logstash.runner]

Elastic Stack Logstash
1

Hello.

I'm using Logstash 5.5 Elastic 5.5 Kibana 5.5 X-Pack 5.5
logstash - ssl - elastic

Here's the error:

[FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: Setting "xpack.monitoring.elasticsearch.ssl.ca" must be a String. Received: ["/etc/logstash/ca.cert.pem"] (Array)>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:228:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:403:invalidate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:413:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:191:inset'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:64:in set_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:83:inmerge'", "org/jruby/RubyHash.java:1342:in each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:83:inmerge'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:135:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:244:inexecute'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:209:inrun'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:71:in(root)'"]}

Config logstash.yml (grep -v "^ #"):

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.url: "https://test:9200"
xpack.monitoring.elasticsearch.username: "logstash_internal"
xpack.monitoring.elasticsearch.password: "test"
xpack.monitoring.elasticsearch.ssl.ca: [ "/etc/logstash/ca.cert.pem" ]
path.data: /var/lib/logstash
path.config: /etc/logstash/conf.d
path.logs: /var/log/logstash

Config /etc/logstash/conf.d/logstash.conf

output {
elasticsearch {
user => "logstash_internal"
password => "test"

    #ssl => true
    #cacert => "/etc/logstash/ca.cert.pem"

    document_type => "nginx_logs"

    hosts => ["https://test:9200"]
    #hosts => ["test:9200"]

    index => "logstash-%{+YYYY.MM.dd}"

}
}

Pls HELP!

2

That is not a string, it is an array, so remove the brackets.

2 Likes
3

Thanks, but new error :

[2017-09-14T00:11:58,390][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#<LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError: No Available connections>}
[2017-09-14T00:11:58,390][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>16}
[2017-09-14T00:12:02,872][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://logstash_internal:xxxxxx@logstashtest:9200/, :path=>"/"}
[2017-09-14T00:12:02,943][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://logstash_internal:xxxxxx@logstashtest:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [https://logstash_internal:xxxxxx@logstashtest:9200/][Manticore::ClientProtocolException] PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"}
[2017-09-14T00:12:07,946][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://logstash_internal:xxxxxx@logstashtest:9200/, :path=>"/"}

4

Did you check that?

5

https://logstashtest:9200/

{
"name" : "7VVpZ3c",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "0JjM1_-2TMGXExMW_P_kFA",
"version" : {
"number" : "5.5.1",
"build_hash" : "19c13d0",
"build_date" : "2017-07-18T20:44:24.823Z",
"build_snapshot" : false,
"lucene_version" : "6.6.0"
},
"tagline" : "You Know, for Search"
}

6

That does not match the URL you have in your config.

7

sorry, i change before send

conf :
...
xpack.monitoring.elasticsearch.url: "https://logstashtest:9200"
...

and:

...
hosts => ["https://logstashtest:9200"]
...

8

I have deleted comments on the lines:
ssl => true
cacert => "/etc/logstash/ca.cert.pem"

output {
elasticsearch {
user => "logstash_internal"
password => "test"

    ssl => true
    cacert => "/etc/logstash/ca.cert.pem"

    document_type => "nginx_logs"

    hosts => ["https://logstashtest:9200"]
    #hosts => ["logstashtest:9200"]

    index => "logstash-%{+YYYY.MM.dd}"

}

Now hangs on stage:

[2017-09-14T00:28:20,731][INFO ][logstash.pipeline ] Pipeline .monitoring-logstash started
[2017-09-14T00:28:20,770][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[https://logstash_internal:xxxxxx@logstashtest:9200/]}}
[2017-09-14T00:28:20,773][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://logstash_internal:xxxxxx@logstashtest:9200/, :path=>"/"}
[2017-09-14T00:28:20,887][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"https://logstash_internal:xxxxxx@logstashtest:9200/"}
[2017-09-14T00:28:20,891][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-09-14T00:28:20,973][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"default"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-09-14T00:28:21,013][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["https://logstashtest:9200"]}
[2017-09-14T00:28:21,459][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125}
[2017-09-14T00:28:22,076][INFO ][logstash.pipeline ] Pipeline main started
[2017-09-14T00:28:22,218][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

9

That's not hung, it's running.

10

Thanks. It's work! =)

It was necessary to remove the parentheses of the array []
Did according to your instructions
https://www.elastic.co/guide/en/x-pack/current/monitoring-logstash.html
xpack.monitoring.elasticsearch.ssl.ca: [ "/path/to/ca.crt" ]

11

We'll get that updated :slight_smile:

1 Like
12

Thank you for your prompt help!

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.