Unassigned Shards on ELK cluster without Replicas

[zenzedoon (Zied BEN SALEM)]

Hello Community , i have an ELK cluster with 3 servers (1 hot - 1 warm & 1 cold) the cluster dosn't support the replicas mode, my problem is , i found some unassigned shards with multiple explanation for the same shards , example :
-"the cluster has unassigned shards and cluster setting [cluster.routing.allocation.allow_rebalance] is set to [indices_all_active]"

• "index has a preference for tiers [data_warm,data_hot] and node does not meet the required [data_warm] tier"
• a copy of this shard is already allocated to this node [[iss-prod-windowsserver-winlog-000002][0], node[XFcdv8-9Tu2WRZouGtIcuA], [P], s[STARTED], a[id=q_F58vFsSOignowFzvfHtg]]
  
  
  
  
  
  

Did any one knows how to reassigne them ??

[angelo]

One of your screenshots indicates that 1 of your nodes at some point disconnected from the cluster (NODE_LEFT) - assuming it has re-joined and the cluster simply gave up trying to reassign shards, you can use the cluster reroute API call POST /_cluster/reroute to retry allocation of all unassigned shards and see if that gets it sorted.

If not, you can explicitly check the reason why it won't or can't assign using allocation explain API for one of the failing shards from your _cat/shards output, ex:

GET _cluster/allocation/explain
{
  "index": "iss-prod-windowsserver-winlog-000002",
  "shard": 1,
  "primary": false
}

The output should run through the 3 nodes. Two of the messages you noted are expected:

• "index has a preference for tiers [data_warm,data_hot] ..." This will be the cold node
• "a copy of this shard is already allocated ..." This will most likely be the node holding the primary copy of the same shard

The remaining node is what will be of interest to determine why it won't accept/create the replica copy that is required. Hopefully that gives you some pointers on how to proceed.

[zenzedoon (Zied BEN SALEM)]

The reroute command dosen't make any modification still have the same shards unassigned , when i run the cmd _cluster/allocation/explain i get this:
for hot node : "the cluster has unassigned shards and cluster setting [cluster.routing.allocation.allow_rebalance] is set to [indices_all_active]"
for hot node : index has a preference for tiers [data_warm,data_hot] and node does not meet the required [data_warm] tier
for cold node: index has a preference for tiers [data_warm,data_hot] and node does not meet the required [data_warm] tier
####### PS: the replica mode not working on my cluster ###########

[angelo]

Is that a typo (?) - you have the hot node listed twice but nothing for the warm node? So it looks like we have it backwards and this index is now supposed to be on the 'data_warm' tier with 1 replica and since you only have on warm node, it cannot assign the replica. If that is the case, you either will have to add more warm nodes or simply drop the replica for this index once it moves to the warm tier.