Kibana Cannot open monitoring dashboard because remote cluster is not accessable

I have a ELK 8.4.3 Main Cluster
I have ELK self-monitoring enabled on this cluster

another ELK 8.4.1 Remote Cluster, on a separate machine, which is not online all the time
I only use this cluster to offload data from main cluster

The problem only occurs when the remote cluster is offline.

When I try to access the monitoring dashboard on the Main Cluster, I got nothing and an error.
The error logs are shown in the screenshot

I think remote cluster does not have anything to do with self-monitoring, but this is preventing me from using it

I feel that there is some issue with name resolving/connectivity/route tables.
Because it is giving "No route to host".
Try to check whether that IP(x.x.x.66) port 9300 is connecting.

The problem only occurs when the remote cluster is offline.

Hi @edwardcccc ,

Is the Remote set with skip_unavailable: true?

If it is and you're still seeing failures you may want to set the tcp retries on the Main cluster OS as shown on TCP retransmission timeout | Elasticsearch Guide [8.4] | Elastic

As long as the Main cluster can respond to a query when the Remote cluster is down, the stack monitoring UI should load.

Alternatively if the .monitoring-* indices all stay on the Main cluster, you can set monitoring.ui.ccs.enabled: false (reference) which will make Stack Monitoring UI load using only indices from the Main cluster.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.