# Metricbeat events not indexed in Elasticsearch

**URL:** <https://discuss.elastic.co/t/metricbeat-events-not-indexed-in-elasticsearch/274998>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [June 4, 2021, 4:00pm UTC](https://discuss.elastic.co/t/metricbeat-events-not-indexed-in-elasticsearch/274998 "2021-06-04T16:00:42Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hugues\_Bernard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hugues_bernard/32/5948_2.png) [@Hugues\_Bernard](https://discuss.elastic.co/u/Hugues_Bernard)\
**Post date:** [June 4, 2021, 4:00pm UTC](https://discuss.elastic.co/t/metricbeat-events-not-indexed-in-elasticsearch/274998/1 "2021-06-04T16:00:42Z")

</div>

I configured metricbeat on one node of a 3 nodes elasticsearch cluster. Metricbeat output is set to a separate elasticsearch on a dedicated VM used for monitoring data only. Metricbeat is sending events every 10 seconds without any error, as seen in its log set in debug level (log says `PublishEvents: 1 events have been published to elasticsearch in 248.464504ms.`). However, on the monitoring elasticsearch node, no data is ever inserted in the metricbeat index. `docs.count` stays `0`, as reported by `_cat/indices`:

```
curl "$(hostname):9200/_cat/indices?v&pretty"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open metricbeat-7.9.1-2021.06.04-000001 nKxgLXjlQXyRf0bkdTUWnQ 1 1 0 0 208b 208b
green open .monitoring-es-7-mb-2021.06.04 H8-4y7VESXKtWrgcn0aRyQ 1 0 7 0 121.1kb 121.1kb
green open .apm-custom-link arYGemzhQbG0xK70_VfiHQ 1 0 0 0 208b 208b
green open .kibana_task_manager_1 P6t1VznFTbucXiwYpo14sg 1 0 6 75 45.5kb 45.5kb
green open .kibana-event-log-7.9.1-000001 bx5yWdw4SoyXnxDshPftSQ 1 0 1 0 5.5kb 5.5kb
green open .apm-agent-configuration YWGN5b3UTC-sSwKtfwJixQ 1 0 0 0 208b 208b
green open .kibana_1 TDF9qiolTAuF8O_-PsglDg 1 0 15 0 10.4mb 10.4mb

```

Kibana is also installed and running fine on the monitoring node, and also shows no data in the metricbeat index.

I see no errors reported anywhere. `metricbeart.yml` is damn simple:

```
output.elasticsearch:
  # monitoring elasticsearch node
  hosts: ["http://<obfuscated>:9200"] 

processors:
  - add_host_metadata: ~

logging.level: debug
logging.to_files: true
logging.files:
  path: /var/log/metricbeat
  name: metricbeat

```

The problem is probably on the elasticsearch monitoring node side but I am out of ideas. The elasticsearch installation is also very simple, and I made sure the exact same versions of all components are used.

Have you any idea/suggestion what could be wrong ?  
Thanks already for your help.  
Hugues

---

<div class="post-metadata">

**Author:** ![angelo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/angelo/32/61325_2.png) [@angelo](https://discuss.elastic.co/u/angelo)\
**Post date:** [June 4, 2021, 7:51pm UTC](https://discuss.elastic.co/t/metricbeat-events-not-indexed-in-elasticsearch/274998/2 "2021-06-04T19:51:37Z")

</div>

In newer versions of Elasticsearch, a refresh is not done unless the index has received a search request ... and subsequently, the stats may not be getting updated to show the current doc count in the index. It sounds like this may be the case here.

You should be able to confirm by either calling the `_search` API or the `_refresh` [API](https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-refresh.html) on that index and then retry the `_cat/indices` call to see the updated doc counts. The `_search` API should also report the updated total hits as well in the response assuming it returns the default 10 docs.

---

<div class="post-metadata">

**Author:** ![Hugues\_Bernard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hugues_bernard/32/5948_2.png) [@Hugues\_Bernard](https://discuss.elastic.co/u/Hugues_Bernard)\
**Post date:** [June 10, 2021, 11:36am UTC](https://discuss.elastic.co/t/metricbeat-events-not-indexed-in-elasticsearch/274998/3 "2021-06-10T11:36:14Z")

</div>

Thanks Angelo for your reply.  
I was fooled by the "metricbeat\*" index: it is not the one that receives monitoring events. The one to look at is ".monitoring\_es-\*".  
The root cause why no monitoring data was visible in Kibana is the fact that metricbeat was not running on the master node of the monitored cluster, but only on non-master nodes. Until metricbeat is set-up and running on a master node, Kibana dashboard seems not able to display cluster monitoring data.

For info: I found out that reading comments on [kibana show elasticsearch no monitoring data , but exists index in elasticsearch · Issue #22483 · elastic/kibana · GitHub](https://github.com/elastic/kibana/issues/22483) :

> The `cluster_stats` document is produced by the elected master node of the monitored cluster, and if it's not present the UI does not recognize other data because it needs the `cluster_stats` to represent the cluster. We're looking at ways to address this issue separately.

Thanks for your help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 8, 2021, 1:36pm UTC](https://discuss.elastic.co/t/metricbeat-events-not-indexed-in-elasticsearch/274998/4 "2021-07-08T13:36:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.