Independant monitoring cluster

Elastic Stack Elasticsearch
1

Unable to create a separate cluster for Monitoring the Elastic stack.

I have tried configuring it via both MetricBeat as well as HTTP exporters, but I do not see the monitoring information for my production cluster. All I see is the monitoring stats for the monitoring cluster itself.

I am running the trial license on the monitoring cluster, and basic on the production cluster.

2

Using metricbeat, I am able to get metric information about Elasticsearch, but it is not visible in the monitoring UI of Kibana

3

Hi @NerdSec,

what exactly do you want to do? Also, this is the Elasticsearch area and this sounds more like a Kibana issue...

Anyway, with the basic license Kibana will only be able to monitor one Elasticsearch cluster, the one to which the Elasticsearch node that Kibana is configured to connect to belongs.

Why would you like to have a separate monitoring cluster? There is probably a good reason but I can't think of one...

Also, which parts of Elastic Stack do you want to monitor? You might have to enable monitoring from some parts e.g. for Logstash

Unlike X-Pack monitoring for Elasticsearch and Kibana, there is no xpack.monitoring.collection.enabled setting on Logstash. You must use the xpack.monitoring.enabled setting to enable and disable data collection.

Basic Elasticsearch and Kibana monitoring is enabled by default I think...

4

Can I not forward that data to a separate elasticsearch instance? We are running a prod setup and want to monitor this regularly, till we get a gold/platinum license.
Also, there is a lot of data already on the cluster and it just clutters everything up. Spaces has solved the issue to some extent, but it is recommended in the guide to have a separate monitoring instance.

Monitoring in a production environment | Elastic Stack Overview [7.4] | Elastic

The issue is only for elasticsearch and kibana.

5

The issue is resolved. For the sake of others who would like to do this, here are the settings I made:

My production cluster has the following cluster settings:

{
  "persistent" : {
    "xpack" : {
      "monitoring" : {
        "elasticsearch" : {
          "collection" : {
            "enabled" : "true"
          }
        },
        "collection" : {
          "enabled" : "true"
        }
      }
    }
  },
  "transient" : { }
}

My monitoring cluster has the following cluster settings:

{
  "persistent" : {
    "xpack" : {
      "monitoring" : {
        "elasticsearch" : {
          "collection" : {
            "enabled" : "false"
          }
        },
        "collection" : {
          "enabled" : "true"
        }
      }
    }
  },
  "transient" : { }
}

And I have configure http exporters in elasticsearch.yml to point to my Monitoring cluster.

I have then configured beats and logstash to send the monitoring data to my Production cluster, and not the Monitoring cluster.

Also, I have disabled kibana monitoring the kibana.yml.

xpack.monitoring.kibana.collection.enabled: false

Post this and deleting all the existing .monitoring* indices on my Monitoring cluster, I was able to view the monitoring data in the Kibana UI.

2 Likes
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.