Logstash missing from monitoring after 5.6.2 upgrade

[flalar]

After upgrading elasticsearch, kibana, logstash and corresponding x-pack plugins, Logstash monitoring is missing from Kibana.

When looking at the monitoring indexes it seems the .monitoring-logstash-6-2017.09.27 index is not beeing updated since the upgrade.

Found this https://www.elastic.co/guide/en/x-pack/current/monitoring-troubleshooting.html but dont have x-pack security enabled so does not work.

Any tips to what is wrong?

[cutch69 (James)]

let me know what you find.

right now i am adding the remove xpack security on all my nodes and rebooting.

if that dosent work. i will roll back to a snapshot before i did the upgrade on the logstash servers.

[Sam_Chen]

try to set below config in logstash.yaml if security disabled.

xpack.monitoring.elasticsearch.username: "logstash_system"
xpack.monitoring.elasticsearch.password: "changeme"

[flalar]

@Sam_Chen that worked like a charm!!! Thanks

[cutch69 (James)]

so i though by disabling secuirty in the es.yml and kibana.yml it would disable it in logstash. was this a new added feature of 5.6.2

[pickypg (Chris Earle)]

Hi @Benjamin_Cuthbert/ @flalar

There was a bug in 5.6.2, which did not exist in 5.6.0 or 5.6.1, of X-Pack monitoring for Logstash , which makes forces it to require a password even when you are not using authentication with Elasticsearch. There should be a subtly logged error message during the startup of Logstash to that effect.

You must set the password using the xpack.monitoring.elasticsearch.password in logstash.yml

This has been fixed for the forthcoming release of 5.6.3.

Setting any password is enough to satisfy this requirement. For example:

# logstash.yml
xpack.monitoring.elasticsearch.password: anypassword

Until Elasticsearch 6.0, all passwords default to "changeme", so this is a solid default to use there as well, but do beware that Elasticsearch 6.0 is going to remove all default passwords.

Hope that helps,
Chris