Unnasigned shards for replica

Elastic Stack Elasticsearch
1

Hi
How can I reassign (UNASSIGNED) shards for replica ?

logstash-prod1-2022.06.21 3     p      STARTED     2833524 es_data_ssd_5_1
logstash-prod1-2022.06.21 3     r      UNASSIGNED

I have tried through below req:

POST /_cluster/reroute
{
  "commands": [

    {
      "allocate_replica": {
        "index": "logstash-prod1-2022.06.21", "shard": 3,
        "node": "es_data_ssd_5_1"
      }
    }
  ]
}

but it got the error

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "[allocate_replica] allocation of [logstash-prod1-2022.06.21][3] on node {es_data_ssd_5_1}{XMyLrsVFSJiKSs-yUhZkgA}{-gwnp-iZTYKaMlpgb6zBFQ}{10.0.9.102}{10.0.9.102:9300}{hs}{rack_id=rack_one, xpack.installed=true} is not allowed, reason: [YES(shard has failed allocating [5] times but [5] retries are allowed)][YES(primary shard for this replica is already active)][YES(explicitly ignoring any disabling of allocation due to manual allocation commands via the reroute API)][YES(can allocate replica shard to a node with version [8.1.0] since this is equal-or-newer than the primary version [8.1.0])][YES(the shard is not being snapshotted)][YES(ignored as shard is not being recovered from a snapshot)][YES(this node is not currently shutting down)][YES(neither the source nor target node are part of an ongoing node replacement (no replacements))][YES(node passes include/exclude/require filters)][NO(a copy of this shard is already allocated to this node [[logstash-prod1-2022.06.21][3], node[XMyLrsVFSJiKSs-yUhZkgA], [P], s[STARTED], a[id=lPEKLoF2QIK6n6HCChh6mw]])][YES(enough disk for shard on node, free: [1.3tb], shard size: [0b], free after allocating shard: [1.3tb])][YES(below shard recovery limit of outgoing: [0 < 2] incoming: [0 < 2])][YES(total shard limits are disabled: [index: -1, cluster: -1] <= 0)][NO(there are [2] copies of this shard and [3] values for attribute [rack_id] ([rack_one, rack_three, rack_two] from nodes in the cluster and no forced awareness) so there may be at most [1] copies of this shard allocated to nodes with each value, but (including this copy) there would be [2] copies allocated to nodes with [node.attr.rack_id: rack_one])][YES(index has a preference for tiers [data_content] and node has tier [data_content])][YES(shard is not a follower and is not under the purview of this decider)][YES(decider only applicable for indices backed by searchable snapshots)][YES(this decider only applies to indices backed by searchable snapshots)][YES(decider only applicable for indices backed by searchable snapshots)][YES(this node's data roles are not exactly [data_frozen] so it is not a dedicated frozen node)]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "[allocate_replica] allocation of [logstash-prod1-2022.06.21][3] on node {es_data_ssd_5_1}{XMyLrsVFSJiKSs-yUhZkgA}{-gwnp-iZTYKaMlpgb6zBFQ}{10.0.9.102}{10.0.9.102:9300}{hs}{rack_id=rack_one, xpack.installed=true} is not allowed, reason: [YES(shard has failed allocating [5] times but [5] retries are allowed)][YES(primary shard for this replica is already active)][YES(explicitly ignoring any disabling of allocation due to manual allocation commands via the reroute API)][YES(can allocate replica shard to a node with version [8.1.0] since this is equal-or-newer than the primary version [8.1.0])][YES(the shard is not being snapshotted)][YES(ignored as shard is not being recovered from a snapshot)][YES(this node is not currently shutting down)][YES(neither the source nor target node are part of an ongoing node replacement (no replacements))][YES(node passes include/exclude/require filters)][NO(a copy of this shard is already allocated to this node [[logstash-prod1-2022.06.21][0], node[XMyLrsVFSJiKSs-yUhZkgA], [P], s[STARTED], a[id=lPEKLoF2QIK6n6HCChh6mw]])][YES(enough disk for shard on node, free: [1.3tb], shard size: [0b], free after allocating shard: [1.3tb])][YES(below shard recovery limit of outgoing: [0 < 2] incoming: [0 < 2])][YES(total shard limits are disabled: [index: -1, cluster: -1] <= 0)][NO(there are [2] copies of this shard and [3] values for attribute [rack_id] ([rack_one, rack_three, rack_two] from nodes in the cluster and no forced awareness) so there may be at most [1] copies of this shard allocated to nodes with each value, but (including this copy) there would be [2] copies allocated to nodes with [node.attr.rack_id: rack_one])][YES(index has a preference for tiers [data_content] and node has tier [data_content])][YES(shard is not a follower and is not under the purview of this decider)][YES(decider only applicable for indices backed by searchable snapshots)][YES(this decider only applies to indices backed by searchable snapshots)][YES(decider only applicable for indices backed by searchable snapshots)][YES(this node's data roles are not exactly [data_frozen] so it is not a dedicated frozen node)]"
  },
  "status" : 400
}

after that it was also increased index.allocation.max_retries from 5 to 7

but once again got error

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "[allocate_replica] all copies of [logstash-prod1-2022.06.21][3] are already assigned. Use the move allocation command instead"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "[allocate_replica] all copies of [logstash-prod1-2022.06.211][3] are already assigned. Use the move allocation command instead"
  },
  "status" : 400
}

What's wrong?

2

Ok already found the root cause: after setting null for allocation and enable once again it was solved allocation

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.