Incorrect number of shard, what is the possible reason?

[mmk1995 (MatthewMak)]

Hello,

The default number of shard is 5 in the yml setting. I also didn't specify any template for all logstash config.
Most of the index are with correct number of shards,
However, some index appears to be only 1 primary shard.
I search all over the internet and cannot find relevant problem.
What could be the possible reason?

[Wolfram_Haussig]

Hello Matthew,

Did you already check if there is an index template for this index where the setting is overwritten? Or could it be that the indexes where created before the setting was changed(The number_of_shards setting only applies at creation date).

Index templates can be checked by the ElasticSearch API:

GET _template

Best regards
Wolfram

[dadoonet (David Pilato)]

Did you upgrade elasticsearch version?
Old versions of elasticsearch were using 5 shards by default. New versions are using 1 shard. Looks like new indices have 1 shard.

[Christian_Dahlqvist]

Given the size of your indices having a single primary shard seems much more appropriate than having 5.

[mmk1995 (MatthewMak)]

Yes! that shoul be the case. Why would the default shard change from 5 to 1?
How large in size should be appropriate to have more shards?

[mmk1995 (MatthewMak)]

what size should be more appropriate per shard?

[Christian_Dahlqvist]

I would recommend aiming for an average shard size of around at least 10GB. If you have a long retention period you could go with a single primary shard and weekly or monthly indices rather than daily. This blog post contains additional details.