Index templates across all nodes in Elasticsearch cluster

Elastic Stack Elasticsearch
1

Hi everyone, I wanted to clarify a couple of things about clustering:

I have a cluster with 3 master+data nodes and 1 co-ordinating node (also running Kibana). I am using Logstash to balance and push data to the master+data nodes directly by specifying their IP addresses in the Elasticsearch output plugin.

Let's say I need 2 primary shards per index across the cluster - from my understanding there's 3 ways I could do this:
(I am not going with specifying index settings as Logstash will be creating new indices on a monthly basis)

  1. Add number_of_shards: 2 to the default logstash-* template - on all 3 nodes. This would also require all indices to be prefixed with 'logstash'.
  2. Create another template with a higher order containing number_of_shards: 2 which corresponds to all indices - * (still running through the logstash-* template first).
  3. A custom template containing the shard settings, corresponding to the indices that I will push into from Logstash.

Which of these is the recommended way to do it?

I have also come across another source stating that creating index templates on a master eligible node would sync that template across all nodes?
If true, I could run one of the options above on one of the master-eligible nodes. If false - do I need to manually create this template on all nodes before Logstash starts sending data?

Thank you!

2

Which of these is the recommended way to do it?

I would recommend the second option, create a template matching * (or whatever pattern) that sets the number of primary shards to 2. Really though, they are all valid solutions for this particular case.

I have also come across another source stating that creating index templates on a master eligible node would sync that template across all nodes?

That's correct, you only need to create the template by sending the request once (to any node), it will be available on all nodes in the cluster.

3

Sounds great. Thank you!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.