Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster

Elastic Stack Elasticsearch
1

Shards are unassigned


s&pretty"
{
  "persistent" : {
    "cluster.routing.allocation.enable" : "all",
    "indices.recovery.max_bytes_per_sec" : "50mb"
  },
  "transient" : {
    "cluster.routing.allocation.enable" : "all"
  }
}


Allocations are enabled

> {
>   "persistent" : {
>     "cluster.routing.allocation.enable" : "all"
>   }
> }
> '
{
  "acknowledged" : true,
  "persistent" : {
    "cluster" : {
      "routing" : {
        "allocation" : {
          "enable" : "all"
        }
      }
    }
  },
  "transient" : { }
}

Cluster are in RED state and i have unassigned shards

{
  "cluster_name" : "k8s-logs",
  "status" : "red",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 0,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 14,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 0.0
}
2

What does Cluster allocation explain API | Elasticsearch Guide [8.5] | Elastic say?

3

It says Elasticsearch isn't allowed to allocated shards to node.

{
  "note" : "No shard was specified in the explain API request, so this response explains a randomly chosen unassigned shard. There may be other unassigned shards in this cluster which cannot be assigned for different reasons. It may not be possible to assign this shard until one of the other shards is assigned correctly. To explain the allocation of other shards (whether assigned or unassigned) you must specify the target shard in the request to this API.",
  "index" : "cs.elastic.cs_policy",
  "shard" : 0,
  "primary" : true,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2022-11-16T11:55:49.507Z",
    "last_allocation_status" : "no"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there."
4

It says more than that, but you've deleted all the useful info which makes it impossible for us to help. The very next sentence tells you what to do anyway:

Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there."

5 
Hi David,

Thank you for reverting back for my query. I ran "http://<host>:9200/_cluster/allocation/explain?pretty"

which gave me the Output as below

{
  "note" : "No shard was specified in the explain API request, so this response explains a randomly chosen unassigned shard. There may be other unassigned shards in this cluster which cannot be assigned for different reasons. It may not be possible to assign this shard until one of the other shards is assigned correctly. To explain the allocation of other shards (whether assigned or unassigned) you must specify the target shard in the request to this API.",
  "index" : "cs.elastic.cs_policy",
  "shard" : 0,
  "primary" : true,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2022-11-16T11:55:49.507Z",
    "last_allocation_status" : "no"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there."
}

'''

I am really not sure why the indexes are not assigned to shards. which made elastic search health red. is there an additional step that needs to be carried to enable shards. ?
6

Is that really the full output? If so, your cluster must have no data nodes.

7

@DavidTurner - No, we have a 3 node cluster. And yes that's the full output

PFB

http://<host>:9200/_cat/nodes

10.XX.Xx.100 49 1 0 1.08 1.63 1.54 m - es-cluster-0
10.XX.XX.56  46 1 0 0.82 1.04 1.01 m * es-cluster-1
10.XX.XX.116 20 1 0 0.77 1.29 1.30 m - es-cluster-2

Do we need to enable shards for node or does it happen dynamically, can you help me with any API call that i can make to get more details.

Few more tested commands

curl -X GET "http://<host>:9200/_cluster/settings?flat_settings&pretty"
{
  "persistent" : {
    "cluster.routing.allocation.enable" : "all",
    "indices.recovery.max_bytes_per_sec" : "50mb"
  },
  "transient" : {
    "cluster.routing.allocation.enable" : "all"
  }
}

curl -X GET "http://<host>:9200/_cat/shards?v=true&h=index,shard,prirep,state,node,unassigned.reason&s=state&pretty"
index                                                         shard prirep state      node unassigned.reason
cs.elastic.cs_group                                           0     p      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_group                                           0     r      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_group                                           0     r      UNASSIGNED      REPLICA_ADDED
.ds-.logs-deprecation.elasticsearch-default-2022.11.16-000001 0     p      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_claim                                           0     p      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_claim                                           0     r      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_claim                                           0     r      UNASSIGNED      REPLICA_ADDED
cs.elastic.cs_client                                          0     p      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_client                                          0     r      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_client                                          0     r      UNASSIGNED      REPLICA_ADDED
cs.elastic.cs_policy                                          0     p      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_policy                                          0     r      UNASSIGNED      INDEX_CREATED
cs.elastic.cs_policy                                          0     r      UNASSIGNED      REPLICA_ADDED
.ds-ilm-history-5-2022.11.16-000001                           0     p      UNASSIGNED      INDEX_CREATED
8

All 3 nodes are dedicated master nodes, which means they can not hold any shards. Change the nodes to have the default node roles and the issue should go away.

9

@Christian_Dahlqvist - Thanks for the input. Can you pass me the command to test it please.

10

You need to change it in your elasticsearch.yml files and restart.

11

Thanks @Christian_Dahlqvist . Let me try them.

12

@Christian_Dahlqvist - Thank you for quick assistance. i indeed missed my node configuration. Looks okay now.

health status index                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   cs.elastic.cs_group  EtMsvwv4Rd6h3yrYE8JTag   1   2        149            0     76.1kb         29.6kb
green  open   cs.elastic.cs_claim  1BI2C4XZRce9jLUsvTXyOA   1   2         44            0     62.8kb         36.4kb
green  open   cs.elastic.cs_client ZLJ857hOTbex-wseaNQegQ   1   2        117            0       27kb         13.8kb
green  open   cs.elastic.cs_policy d17M3EjhR2aDWni0H7BJXA   1   2        143            0     77.3kb         35.9kb
13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.