Monitoring remote Elasticsearch performance with metricbeats and Kibana "Monitoring"

[sgreszcz (Stephen Greszczyszyn)]

Running elasticsearch/kibana (central) and metricbeats version 6.3

I've just enabled metricbeats to monitor 6 edge servers that are running local elasticsearch-based application instances. metricbeats is shipping the metrics to a central elastic/kibana server running x-pack with a license.

I have monitoring enabled and I can see the Kibiana "monitoring" dashboards for the central Elasticsearch database metrics, but for some reason I cannot see any of my remote elasticsearch database metrics in the kibana "monitoring" dashboard.

I can see all the remote elasticsearch metrics in the metricbeat-* index.

Should Kibana be "smart" enough to add the remote elasticsearch node to the "monitoring" dashboards or do I need to build a custom dashboard for the remote metricbeats monitored elasticsearch nodes?

My metricbeats config is:

#------------------------------- ElasticSearch Module -------------------------
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-module-elasticsearch.html

- module: elasticsearch
  metricsets:
  - node
  - node_stats
  period: 10s
  hosts: ["localhost:9200"]
  #username: "user"
  #password: "secret"

[ruflin]

At the moment the data format and index for the data collect by Metricbeat and the internal X-Pack monitoring are very different. So if you want to use the Metricbeat data you have to build your own dashboards on top of it as X-Pack Monitoring is not aware of the data.

Do make use of the X-Pack Monitoring you must configure each of your Elasticsearch Clusters to report to a remote monitoring cluster.

We are working on ideas on how Metricbeat can be used for this in the future.

[sgreszcz (Stephen Greszczyszyn)]

@ruflin - thanks for the feedback!

Do the remote elasticsearch nodes also need an x-pack license to take advantage of the kibana x-pack monitoring too? What type of configurations would you need to do on the remote nodes? Documentation anywhere?

[ruflin]

As you use 6.3 x-pack basic is already bundled with it and you already have a license.

The configuration options to send data to the remove cluster you can find here: https://www.elastic.co/guide/en/elasticsearch/reference/6.3/monitoring-settings.html#http-exporter-settings

[sgreszcz (Stephen Greszczyszyn)]

Thanks @ruflin!