Hi,
I'm setting up two elastic search clusters with helm charts. The clusters are up and running fine on Kubernetes.
Now I would like to use metricbeat to ship ES metrics from cluster 1 to cluster 2 (monitoring cluster).
Currently, both clusters have 3 dedicated master, client and data nodes.
I deployed one metricbeat container, pulling metrics from ES1 and shipping them to ES2. I would like to confirm how it is supposed to be deployed in a container environment.
Do I need one metricbeat container for each elastic search node (3 master/data/client)? Or can a single metricbeat pull metrics from my whole cluster, by pointing the hosts property of my elasticsearch module configuration to the cluster address?
The stack monitoring page shows that my cluster has 9 nodes, but it doesnt list all of them when I go to the nodes page.
Having metricbeat pull metrics from the cluster rather than individual nodes would make things easier when scaling the ES cluster, so I dont have to provision additional metricbeat containers along with it. I'm not sure if this is how the solution was designed though.
Thanks