Hi everyone, I wanted to clarify a couple of things about clustering:
I have a cluster with 3 master+data nodes and 1 co-ordinating node (also running Kibana). I am using Logstash to balance and push data to the master+data nodes directly by specifying their IP addresses in the Elasticsearch output plugin.
Let's say I need 2 primary shards per index across the cluster - from my understanding there's 3 ways I could do this:
(I am not going with specifying index settings as Logstash will be creating new indices on a monthly basis)
- Add number_of_shards: 2 to the default logstash-* template - on all 3 nodes. This would also require all indices to be prefixed with 'logstash'.
- Create another template with a higher order containing number_of_shards: 2 which corresponds to all indices - * (still running through the logstash-* template first).
- A custom template containing the shard settings, corresponding to the indices that I will push into from Logstash.
Which of these is the recommended way to do it?
I have also come across another source stating that creating index templates on a master eligible node would sync that template across all nodes?
If true, I could run one of the options above on one of the master-eligible nodes. If false - do I need to manually create this template on all nodes before Logstash starts sending data?