How many Primary and Replica shards will be allotted for a Dynamic elasticsearch Index (ex:index => "sample-%(YYYY.MM.DD)")

Elastic Stack Elasticsearch
1

Hi,

My logstash is indexing to a dynamic index "sample-%(YYYY.MM.DD)"). I have assigned Primary Shards and Replica shards as 3 and 2 respectively. But when logstash sends to the dynamic index it's creating an index(sample-2017.08.17) with 5 primary shards and 1 replica. Why is it not 3 Primary and 2 Replica?

Thanks in advance..

2

What operations have you performed to make this assignment?

3

Hi Thiago,

I am using the following API,

curl -XPUT 'localhost:9200/sample?pretty' -H 'Content-Type: application/json' -d' { "settings" : { "index" : { "number_of_shards" : 3, "number_of_replicas" : 2 } } } '

4

I understand what is the issue here. What you have created there is an actual index named "sample". But what you need to create is an Index Template so when you create any index that matches "sample-*" it will use the settings from the template.

5

Hi Thiago,

I think I will need to add number_of_replicas field to settings. I don't understand the purpose of mapping, properties, created at blocks. Can you please explain it?

6

Defining a mapping manually is required when you want to fine-tune how Elasticsearch process documents. It is all explained here. But if you just starting with Elasticsearch, you don't need to worry about it, as Elasticsearch can automatically autodetect the mapping when new documents are indexed.

7

But at this point I am not using "Type" concept for indexing. So, am I still good to use Index template API? And can you also let me know the advantage of using Type/Mapping? Once elasticsearch has good amount of documents then I will need to use Type concept?

8

This is not really because of Type (types will be deprecated anyway). You need an index template so every time you create a "sample-*" index then the same settings will be reused.

Your initial problem was that you created 2 indices: an index named "sample" and another one following "sample-%{+YYYY.MM.dd}". That's not how it works, to be able to reuse the settings that you have used when creating "sample" index you need to create an Index Template.

For fully understanding how this works I recommend that you attend the Elastic Stack: Logging eLearning online course

9

Thiago,

Thanks for the complete information :slight_smile:

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.