Logstash X-Pack monitoring troubles

Elastic Stack Logstash
1

Hello,

I'm trying to setup monitoring of some logstash machines, but I'm running into a few issues:

  1. The documentation (https://www.elastic.co/guide/en/x-pack/5.2/monitoring-logstash.html) states that the "xpack.monitoring.elasticsearch.url" setting takes either a string or array. I would like to use the latter (for load balancing purposes, my logstash machines talk to the cluster through nginx). However, if I copy the example setting from the documentation page logstash startup fails with the following error:

[2017-02-01T20:03:17,321][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<ArgumentError: Setting "xpack.monitoring.elasticsearch.url" must be a String. Received: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"] (Array)>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:208:invalidate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:384:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:171:inset'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:61:in set_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:80:inmerge'", "org/jruby/RubyHash.java:1342:in each'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:80:inmerge'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:115:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:210:inexecute'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:183:inrun'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:71:in(root)'"]}`

So does that mean that the documentatio is wrong, or that there is a bug somewhere?

  1. How can I change the index that logstash uses for monitoring data?

  2. I need to use TLS client certificates for authenticating to the cluster, but I can only seem to find options for setting a CA certificat,e not client certificates. What options would I use for client certificates?

Thanks in advance!

2

I'm having the same issue! We switched to a string, but I don't see any monitoring data. :frowning:

EDIT: We solved our issue by using a string and removing any trailing slashes. Good luck!

3

Thank you, however this is not the issue i'm having. It connects just fine when I specity the URL as a string, however, the documentation states you should be able to use an array of URL's. This does not seem to work though.

In other news, I have "solved" my problem with client certificate authentication. It seems the plugin developers neglected to add the option to the plugin at all. So I created two patch files to add it:

Hopefully someone from Elastic will pick up on this and make sure it gets fixed in a future release of X-Pack.

4

With regards to setting the ES hosts to ship metrics to (xpack.monitoring.elasticsearch.url), we only support a single host (string) in 5.2.0. We've updated the documentation to reflect this: https://www.elastic.co/guide/en/x-pack/current/monitoring-logstash.html.

With that said, we'll be adding the ability to pass in an array of hosts soon.

Additionally, we do currently only support truststore (CA cert) and not yet keystore (client cert). I'll make sure to track this one on the Elastic end.

Let us know if you find anything else, thanks!

5

Thank you! It's the same as with the CA certificate setting btw :slight_smile: