We have a large number of shards spread across around 80 daily indexes and growing (5 primary 1 Replica). So far we're clocking around 400 shards. Clients do tend to query data in the older indexes.
We have 5 master data nodes and 2 Coordinating nodes. Master Data Nodes are running on m5x4Large EC2 Instances in AWS. So altogether 7 Nodes including the Coordinating Nodes.
Had a few questions around this:
- Is this a a large number of shards (what is considered large from my stack perspective)?
- When should I merge/shrink indexes? Which is preferable to keep the elastic stack healthy?
- We're starting to see a few EsRejectedExecutionException[rejected execution (queue capacity) exceptions in the nodes. What configuration should I be looking at to solving this?
Looking for any help on this. Thanks!