Monitoring Elasticsearch via another cluster on cloud.elastic.co

Elastic Stack Elasticsearch
1

Hi,
We've setup two clusters, one for the application and one for monitoring via cloud.elastic.co.

The application uses Logstash and Beats as well. We've configured the application cluster to report metrics to the monitoring cluster on the cloud dashboard. We've configured Logstash and beats to report their metrics to the monitoring cluster.

This seems right to us, because we wouldn't want the application cluster to need to be up for us to monitor the Logstash and Beats instances. However, the way the information is presented in Stack Monitoring, as well as a configuration issue we're having with exporters is leading us to wonder with the intention is that in this setup Logstash and Beats should be sending their metrics to the application cluster (which I can see an argument for) to be forwarded to the monitoring one.

Which is right?

Thanks, Dan.

2

Hi Dan,

The way you should do this going forward is to use Metricbeat to collect monitoring data from Logstash and Beats and configure it to ship that data directly to the monitoring cluster. Details on this configuration can be found here: https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-metricbeat.html

3

Hi Mike,

We don't have a problem sending our ES metrics to our monitoring cluster, as cloud.elastic.co allows us to set this up easily. The question is relating to Logstash and Beats monitoring and whether the in-built reporting there should be configured to send their data to which cluster.

The cloud.elastic.co monitoring seems to be a wholesale "any monitoring data" gets forwarded to a different cluster, making sending metrics from LS/Beats to this cluster a possibility, and have them forwarded to the monitoring cluster. The upside being less configuration, the downside being a a lack of redundancy.

In our current setup, where LS/Beats send their monitoring data to the monitoring cluster directly, they get reported in the Kibana UI as belonging to the monitoring cluster, hence the ambiguity and question.

Thanks, Dan.

4

In our current setup, where LS/Beats send their monitoring data to the monitoring cluster directly, they get reported in the Kibana UI as belonging to the monitoring cluster, hence the ambiguity and question.

Are you setting the cluster_uuid value in the monitoring configuration as described here? Use internal collection to send monitoring data | Filebeat Reference [7.5] | Elastic

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.