# Monitor elasticsearch with metricbeat, cannot get "index" metricset

**URL:** https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816
**Category:** Beats
**Tags:** metricbeat
**Created:** [August 5, 2020, 7:28am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816 "2020-08-05T07:28:19Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)
#### Post date: [August 5, 2020, 7:28am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/1 "2020-08-05T07:28:19Z")

</div>

I am running on kubernetes, with elastic Helm, 3 master nodes and 3 data nodes. Metricbeat elastic Helm to run metricbeat.

I configure metricbeat to monitor my 6 nodes ES cluster, and give it as elasticsearch host the master k8s service.

But metricbeat complains "cannot fetch from no-master node", and I dont get the "index" metricset.

What the correct way to get all metricsets?

---

<div class="post-metadata">

### Author: ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)
#### Post date: [August 5, 2020, 8:16am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/2 "2020-08-05T08:16:17Z")

</div>

Okay ! the documentation is lacking this: " Install Metricbeat on each Elasticsearch node in the production cluster".

Why metricbeat is not fetching all nodes automatically?

---

<div class="post-metadata">

### Author: ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)
#### Post date: [August 5, 2020, 8:21am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/3 "2020-08-05T08:21:31Z")

</div>

Finaly we move to [https://github.com/justwatchcom/elasticsearch\_exporter](https://github.com/justwatchcom/elasticsearch_exporter) , with prom/grafana.

I really try to push elastic products in my company, but it's so hard to install compare to prometheus universe 😕

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [August 5, 2020, 3:07pm UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/4 "2020-08-05T15:07:32Z")

</div>

Hi @ebuildy,

I'm sorry to hear it's not working well for you. The architecture of requiring a beat on each node puts an emphasis on reliability. It enables monitoring of nodes even if they're not part of a cluster. This might be of particular importance to diagnose the reason why it's not able to join.

I am seeing "Install Metricbeat on each Elasticsearch node in the production cluster" in the docs. Which version of the docs are you viewing?

---

<div class="post-metadata">

### Author: ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)
#### Post date: [August 6, 2020, 12:14pm UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/5 "2020-08-06T12:14:29Z")

</div>

I have followed the doc at [https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-elasticsearch.html](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-elasticsearch.html).

Well, we use now elasticsearch-exporter for prometheus, this requires a single instance that connect to a single node and work straightaway. And there is a grafana dashboard already (whereas metricbeat is missing a dashboard for elasticsearch metrics .....).

Also, the non-master node error is logged as "debug" level:

> <https://github.com/elastic/beats/blob/44fc8e3302138dd7bd0be8d6ac2ff6754cc63c20/metricbeat/module/elasticsearch/metricset.go#L132>

So it took me a lot of time to find it 😕

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [August 7, 2020, 8:13am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/6 "2020-08-07T08:13:53Z")

</div>

True, this modules' docs are a bit terse. The reason for there not being any dashboards is that the built-in stack monitoring app provides an even more curated UI for monitoring. Have tried using the stack monitoring feature?

---

<div class="post-metadata">

### Author: ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)
#### Post date: [August 7, 2020, 8:23am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/7 "2020-08-07T08:23:31Z")

</div>

Yes, I tried it, really nice.

In my case, I want to monitor multiple ES clusters, with a basic licence 😕

About installing metricbeat on every node, it's really hard, maybe, you should consider to add metricbeat as a sidecar (or at least document it) when using the official elastic Helm chart for kubernetes.

We have finally moved back to prom/grafana with elasticsearch-exporter, because it's more simple and work really fast.

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [August 11, 2020, 9:11am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/8 "2020-08-11T09:11:09Z")

</div>

Yes, that could indeed be convenient for that deployment scenario. I saw you opened an issue about it - thank you!

It's good to see you found a solution that works for you in the meantime. Thanks for sharing with the community.

Have you considered deploying Metricbeat as a daemonset on each node using its helm chart and using the "autodiscover" feature to automatically enable the Elasticsearch module where needed?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 8, 2020, 9:11am UTC](https://discuss.elastic.co/t/monitor-elasticsearch-with-metricbeat-cannot-get-index-metricset/243816/9 "2020-09-08T09:11:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
