Metricbeat elasticsearch module can't monitor [cluster_stats,index,shard] metricsets when xpack.monitoring.exporters is set

[tetsuyasodo]

Hi,
I'm using an ES6.8 3 nodes cluster(cluster A) and an ES6.8 1 node cluster(cluster B).
The cluster B is used for monitoring the cluster A.
I'm exporting the .monitoring index from cluster A to cluster B.
The configuration for exporting is the following:

xpack.monitoring.collection.enabled: true
xpack.monitoring.exporters:
  monitoring_cluster:
    type: http
    host: ["http://mon1:9200"]

I've also installed metricbeat on each node of cluster A and enable the elasticsearch module.
In this module I'd like to fetch the detail metrics by using cluster_stats, index and shard metricsets.

The configuration of metricbeat is the following:

- module: elasticsearch
  metricsets:
    - node
    - node_stats
    - cluster_stats
    - index
    - shard
  period: 10s
  hosts: ["http://localhost:9200"]

However, the metricbeat doesn't fetch these metricsets from cluster A. Only node and node_stats metricsets can be fetched.
I know there is no ".monitoring" indices in cluster A because they are transferred to cluster B. I'm afraid this is cause of this symptom.
Do we need to have their own ".monitoring" indices for the cluster in order to fetch the detail metrics for elaticsearch module?

Thanks for your help
Tetsuya

[mtojek (Marcin Tojek)]

Hi @tetsuyasodo,

could you please share some logs with us (temporarily enable logging)? Maybe it's a typo somewhere.

[tetsuyasodo]

Hi @mtojek ,
Thanks for your comment.
I've found I ran only one metricbeat of three nodes and it is not the master node.
Starting the metricbeat on the master node solved the problem and I can see all the metricsets including cluster_stats, index and shards now.
Your comment let me clarify they should be fetched with this architecture.
Much appreciated!

Tetsuya