I am running a 8 node cluster (ES 6.7.0) of which one node I am using solely for the purpose of monitoring other 7 nodes. The node name for this lets say is monitoring. Now I am trying to monitor other nodes using x-pack (free version). The x -pack creates daily indices. My requirement is that I want these indices which are created by x-pack to have just one shard and no replica and I want the shard to reside on the monitoring node only. To achieve this I am updating the .monitoring-es template with following curl request.
Shard and replica settings are taking effect. But the routing settings are not taking effect. The daily shards are being created on different nodes. How can I tie down the monitoring shard to just one node?
Typically, users set up an independent monitoring cluster to monitoring the Elastic Stack and this is the configuration that Elastic recommends. From what you're describing, though, it sounds like you're trying to use a single cluster where one node monitors the rest of the others? Is there a specific reason that you don't want to use an independent monitoring cluster?
Thanks @Mike_Place for your response, Yes my monitoring node lies in the same cluster. I am trying x-pack monitoring for my nodes. In the x-pack monitoring that comes with the basic license we can only monitor the cluster in which the monitoring node lies. So I had to put this node in the production cluster.
With the basic license, you can actually set up a separate cluster and use it to store your monitoring data. Just now, @shaunak did a quick test with the following steps and confirmed that it does work as expected:
1. Run a dedicated ES cluster for monitoring data, `esmon`
2. Run a separate ES cluster for production data, `esprod`
3. Set `xpack.monitoring.elasticsearch.hosts` in `kibana.yml` to point to an `esmon` node's HTTP address.
4. Setup an `http` exporter on `esprod` to forward data to `esmon`
Did you try to set up a separate cluster and if so, did you encounter an error?
I have actually got busy with some other task at hand. Will try out the settings recommended by @Mike_Place soon. I did encounter an error while trying to setup a separate cluster. Not able to fetch the error now since I have moved away from that setup.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.