X-Pack Monitoring. Data Missing

Elastic Stack Elasticsearch
1

Hi,
I have 2 Logstash nodes and an Elasticsearch Cluster of 3 Nodes. All are configured for X-Pack Monitoring along with Kibana.
I wanted to validate the fail over mechanism for the Elasticsearch cluster configuration, so I stopped one of the Elasticsearch nodes for 20 mins and started it back again. But the monitoring data of all nodes in cluster and also the 2 Logstash instances is missing.

Why is the data of other two Elasticsearch and Logstash nodes missing?

After i bring up the Elasticsearch node, the status changes from red to yellow and after the index syncs up turns green.

Thanks in advance!

2

For ES cluster missing monitoring data when one of the 3 nodes is stopped. Can you verify x-pack was installed on all nodes (bin/elasticsearch-plugin list), and is not disabled in elasticsearch.yml with setting xpack.monitoring.enabled: false?
Regarding logstash not sending monitoring data, can you verify xpack.monitoring.elasticsearch.url setting in logstash.yml contains the URL of all 3 nodes?
Beside this, if this is a reproducible issue when stopping the node again, please check the cluster health (in particular monitoring indices /_cat/indices) is not red when the node is down (should not happen unless indices use 0 replica - if index is red no write operation will be possible on that index)
Also check logs of ES and logstash for more information as there could be information there.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.