Monitoring Not works after some time

Elastic Stack Elasticsearch
1

Hi All ,

My 5 elasticsearch cluster sending monitoring data to another elasticsearch(monitorin server) machine. Kibana is using to visualize 5 node elasticsearch cluster and monioring data.

All configurations are working well .id1 ,1d2,1d3,1d4.1d5
All elasticsearch nodes are shown in kibana. But after sometime kibana showing error message:

""""""""""""

You need to make some adjustments
To run monitoring please perform the following steps

We checked the cluster defaults settings for xpack.monitoring.exporters, and found the reason: Remote exporters indicate a possible misconfiguration: id5.
"""""

2 kibana is using . A load balancer using for connecting to kibana from browser.

My question is monitoring server is worked well for some days( Graphs are shown) . after 5 days this kibana showing error .

If I am restart any kibana the error will change to 1d5 to id1

Thanks

2

Hi @rijinmp,

Can you post the elasticsearch.yml files for each node, as well as the results of:

GET _cluster/settings

for each node too?

Feel free to scrub any sensitive/private information

Thanks

3

Elasticsearch cluster conf:

cluster.name: cluster1
node.name: hostname1
node.master: false
node.data: true
node.ingest: false
search.remote.connect: false
path.data: /opt/data/elasticsearch

path.logs: /var/log/elasticsearch
network.host: 192.168.1.4
http.port: 9200

transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["hostname1", "hostname2","hostname3","hostname4","hostname5"]

discovery.zen.minimum_master_nodes: 2
xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/hostname1.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/hostname1.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/hostname1.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/hostname1.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.collection.enabled: true
xpack.monitoring.collection.enabled: true
xpack.monitoring.exporters:
id1:
type: http
host: ["http://monitoringip:9200"]

1d1 , 1d2, 1d3,1d4,1d5 are using in elasticsearch machines for monitoring .
3 master node and 2 data nodes are using in cluster
IPs and host names are the only difference in other machines