How many shard should i have for index size 450gb

Elastic Stack Elasticsearch
1

Hi

Currently we have one index with following cofiguration.
No of primary shard = 10
no of replica shard = 1
size of index = 450 apporox
no of doc count = 1 billion
rotation of index = weekly

Cluster info:-
no of nodes = 6
coordinating node = 1 (core = 64, heap = 30gb)
data/master eligible node = 3
dedicated data node =2
heap size = 25 gb on all node.
core = 12

so currently, In discovery tab, we can easily fetch last 5 days data but we can't able to fetch last 7 days data.

i tried to increase replica shard but no luck.

can you please suggest us what should be correct no of shard for fetching last 7 days data. or Is there any other setting(like refresh interval etc) that will help me.

2

Hi @Jeetu19,

Replica shards are used in case any primary shard is corrupt. This is just a copy of your shard.

Fetching last 7 days data (filter) and refresh interval are on kibana settings. There mayb overhead problem in your cluster while kibana querying the data to elasticsearch index/shards.

you can look into below that how elasticsearch works and deal with overhead:

In elastic cluster, no of shards should be proportional to the amount of heap you have available, but there is no fixed limit.

For more clarification and information you can refer below blog to reduce the overhead and increase performance of elastic cluster.

Link:

Regards,
Harsh Bajaj