Logstash monitoring with Metricbeat and cluster_uuid

When configuring Logstash monitoring, the Elastic recommended best practice is to create a cluster dedicated to log and metrics collection. One then configures Metricbeat to collect the monitoring data from Logstash using the logstash or logstash-xpack modules. This contains the information on how Metricbeat is to communicate with Logstash. Metricbeat is configured to write the metrics it collects to the monitoring cluster.

All good so far.

However, the metrics collected are reported as being from "Standalone Cluster" cluster when viewing the monitoring cluster through Kibana. This is annoying and counter-intuitive.

Elastic's solution to this is to set monitoring.cluster_uuid in logstash.yml to the UUID of the monitoring cluster. This unfortunately means logstash.yml must have knowledge of the monitoring cluster - which it really cares nothing about. And since Metricbeat can be configured to monitor several Logstash instances, this means every instance must be modified to have knowledge of the monitoring cluster.

This seems like a very poor practice.

The poorness of this practice is further compounded in that Logstash need not interact with Elasticsearch at all. It may output to files, an S3 bucket, or many other things other than Elasticsearch. Imposing knowledge of the monitoring cluster on a Logstash instance seems very bad.

Better would be setting monitoring.cluster_uuid in Metricbeat's logstash or logstash-xpack module, in metrcibeat.yml, or provide some way to link the data cluster with the monitoring cluster so that the monitoring cluster could know the name and other metadata about the data cluster.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.