My indices are having 1 replicas and 3 shards (3x2)
There are some indices having a filter "index.routing.allocation.required._id": id-of-a-certain-node
, blocking the replica shard to allocate on any node due to two restrictions:
- (Decider: filter) Routing filters required the shards be allocated on a specific node
- (Decider: same_shard) Primary and replica shards cannot be on the same node
The routing filters are not applied by human, not sure why these filters exists.
There are also some shards un-allocated due to memory issue, but resolved after increasing memory and JVM heap.
The incidence is resolved, so I won't be able to provide additional error logs, but I'll try my best to answer any question regarding the issue.
Commands used for checking the status:
/_cat/health
/_cat/shards
/_cat/shards/<shard>
/<index>" | jq '."<index>".settings.index.routing'
/_cluster/allocation/explain