We are currently on version 7.17.5 with the full ELK stack and are working on getting ready to jump to 8.x with the help of Upgrade Assistant.
One of the cluster issues is:
Setting [xpack.monitoring.collection.enabled] is deprecated
The Documentation link for that issue says
Using the Monitoring plugin to collect and ship monitoring data is deprecated. Metricbeat is the recommended method for collecting and shipping monitoring data to a monitoring cluster.
Ok so I took a look at Metricbeat:
On this page it says under point 1 that you should set
true. But the issue we are coming from clearly says that this setting is deprecated! I'm slightly confused but keep reading for now....
On the same page under point 2 it says:
Ideally install a single Metricbeat instance configured with
scope: clusterand configure hosts to point to an endpoint (e.g. a load-balancing proxy) which directs requests to the master-ineligible nodes in the cluster.
I would like to focus here on what the proxy should point to: to the master-ineligible nodes in the cluster.
Now let's compare this information with the section about the elasticsearch module from the Metricbeat doc:
scopeis set to
cluster, each entry in the hosts list indicates a single endpoint for a distinct Elasticsearch cluster (for example, a load-balancing proxy fronting the cluster).
There is no mention of the fact that only certain nodes may be reached via this endpoint. Perhaps this is common knowledge but I am a bit unsure and would like some clarification on the subject.
Our situation is as follows:
We have an Elasticsearch cluster with dedicated master, ingest and data nodes. To be clear, these nodes each have exactly one role: either master or ingest or data. Furthermore, there is a load-balancing proxy that distributes all traffic to the two ingest nodes. All other applications use this proxy to communicate with the cluster in some way.
Is that correct now or what should the proxy point to?
(1) Only master-ineligible nodes
(2) All master-ineligible nodes
This does make a difference. We currently have (1) fulfilled but not (2) - for that, the data nodes would also have to be behind the proxy. Which would appear wrong to me somehow.
Is our setup correct and with this setup would a single Metricbeat instance be sufficient to monitor Elasticsearch with
Many thanks in advance for your help!