Prevent co-placement of daily heavy-duty indices

Elastic Stack Elasticsearch
1

Hi
We operate a >100 nodes cluster, the size mainly intended to allow requests on a big temporal depth.
We are daily creating a few separate indices for indexing ( up to ten shard per indice + replicas ).
We notice that shards from these different indices are often allocated to the same node, leading to high resources contention.

Indices are created at the same hour ( midnight).
Any idea how we can have allocation to separate nodes ?

P.s. of course we have limited to 1 shard per node these indices...

2

Hi Cedric, and welcome.

If you don't want particular indices to be co-located then the simplest solution is to define the sets of nodes for each index up-front, using node attributes and shard allocation filtering.

It's not unusual to only want to apply this restriction while the index is actively writing documents. If so, you can remove the allocation filter once the next index becomes active.