# Getting Elasticsearch cluster status as RED

**URL:** <https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-monitoring, docker\
**Created:** [May 17, 2022, 12:11pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939 "2022-05-17T12:11:43Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 12:11pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/1 "2022-05-17T12:11:43Z")

</div>

Hi Community,  
I'm using Elastic stack on k8s and facing ES cluster status as RED but indices are all in green  
below is cluster health status  
[Elasticsearch@Elasticsearch-master-0 ~]$ curl -XGET 'localhost:9200/\_cluster/health?pretty'

```auto
[elasticsearch@elasticsearch-master-0 ~]$ curl -XGET 'localhost:9200/_cluster/health?pretty'
{
  "cluster_name" : "elasticsearch",
  "status" : "red",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 10,
  "active_shards" : 15,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 2,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 88.23529411764706
}

```

anyone please help me out from this

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [May 17, 2022, 12:23pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/2 "2022-05-17T12:23:11Z")

</div>

Hello @Yogendra_Pratap_Sing

This is due to the active\_shards count. Please wait for time to get it assigned.

just execute the below command which will allocate the failed shards

```auto
curl -X POST http://127.0.0.1:9200/_cluster/reroute?retry_failed=true

```

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 12:26pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/3 "2022-05-17T12:26:37Z")

</div>

Thanks @sudhagar_ramesh for your response I hit your command and status is

```auto
acknowledged":true

```

but still getting the same status on clusterHealth.

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 12:29pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/4 "2022-05-17T12:29:40Z")

</div>

This is indices status

```auto
[elasticsearch@elasticsearch-master-0 ~]$ curl -XGET http://localhost:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .monitoring-es-7-2022.05.17 0L1HYNueQF245i3UR6Tljg 1 1 14258 8041 16.8mb 9.5mb
green open filebeat-7.17.3 HmPnIwQBQ9iXuJS3kW3-LA 1 0 148830 0 70.2mb 70.2mb
green open apm-7.12.1-transaction 6e6rUeowQPOLQJaR8mBeHA 1 0 189045 0 100mb 100mb
green open apm-7.12.1-metric 56Xipem5RvCl9c7DHt-NHw 1 0 54095 0 10mb 10mb
green open apm-7.12.1-span MutIu__0Rve4vS72KT6xsg 1 0 215897 0 49.4mb 49.4mb
green open apm-7.12.1-error azc2O_WqT8CpibocRLBbzA 1 0 41 0 415kb 415kb
green open .async-search FgI1zzpTTteRBvC3dQsLaw 1 1 2 0 2mb 1mb

```

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [May 17, 2022, 12:34pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/5 "2022-05-17T12:34:42Z")

</div>

Could you please share the outputs of below command

```auto
GET /_cluster/allocation/explain 

GET /_cat/indices?v

```

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [May 17, 2022, 12:39pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/6 "2022-05-17T12:39:35Z")

</div>

I believe this is a one node cluster.

execute the below command which will make the replicas of each index to zero

```auto
PUT /*/_settings
{
 "index" : {
  "number_of_replicas":0
 }
}

```

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 12:44pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/7 "2022-05-17T12:44:13Z")

</div>

Here it is what you asked please check

```auto
[elasticsearch@elasticsearch-master-0 ~]$ curl -XGET localhost:9200/_cluster/allocation/explain?pretty
{
  "index" : ".ds-ilm-history-5-2022.05.17-000013",
  "shard" : 0,
  "primary" : false,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "CLUSTER_RECOVERED",
    "at" : "2022-05-17T10:59:47.050Z",
    "last_allocation_status" : "no_attempt"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "8S3wqTdGRHq7a7XWWHEyzQ",
      "node_name" : "elasticsearch-master-0",
      "transport_address" : "172.31.36.250:9300",
      "node_attributes" : {
        "ml.machine_memory" : "2147483648",
        "ml.max_open_jobs" : "20",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1073741824",
        "transform.node" : "true"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "replica_after_primary_active",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        },
        {
          "decider" : "throttling",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        }
      ]
    },
    {
      "node_id" : "DbnhybKiRjSc4dgBomSjZw",
      "node_name" : "elasticsearch-master-1",
      "transport_address" : "172.31.30.203:9300",
      "node_attributes" : {
        "ml.machine_memory" : "2147483648",
        "ml.max_open_jobs" : "20",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1073741824",
        "transform.node" : "true"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "replica_after_primary_active",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        },
        {
          "decider" : "throttling",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        }
      ]
    },
    {
      "node_id" : "OLvyXR_rTq-H6DqK0OWceQ",
      "node_name" : "elasticsearch-master-2",
      "transport_address" : "172.31.53.196:9300",
      "node_attributes" : {
        "ml.machine_memory" : "2147483648",
        "ml.max_open_jobs" : "20",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1073741824",
        "transform.node" : "true"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "replica_after_primary_active",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        },
        {
          "decider" : "throttling",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        }
      ]
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [May 17, 2022, 12:46pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/8 "2022-05-17T12:46:37Z")

</div>

Hello @Yogendra_Pratap_Sing

Please execute this command and check the cluster state. Hope this would help you

```auto
PUT /*/_settings
{
 "index" : {
  "number_of_replicas":0
 }
}

```

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 1:06pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/9 "2022-05-17T13:06:00Z")

</div>

@sudhagar_ramesh I hit your command and status is

```auto
"acknowledged":true

```

but still getting the same status on clusterHealth.

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [May 17, 2022, 1:10pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/10 "2022-05-17T13:10:43Z")

</div>

Hello @Yogendra_Pratap_Sing  
i think this index ".ds-ilm-history-5-2022.05.17-000013" is still getting recovered to confirm the same could you execute this command

`GET _cat/recovery/event_tracking?v`

Also this is a system index . hence no need to worry about the cluster health

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 1:12pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/11 "2022-05-17T13:12:56Z")

</div>

> [@sudhagar\_ramesh](#):
>
> GET \_cat/recovery/event\_tracking?v

@sudhagar_ramesh there is no index named like this

```auto
[elasticsearch@elasticsearch-master-0 ~]$ curl -XGET localhost:9200/_cat/recovery/event_tracking?v                                 
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [event_tracking]","resource.type":"index_or_alias","resource.id":"event_tracking","index_uuid":"_na_","index":"event_tracking"}],"type":"index_not_found_exception","reason":"no such index [event_tracking]","resource.type":"index_or_alias","resource.id":"event_tracking","index_uuid":"_na_","index":"event_tracking"},"status":404}

```

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [May 17, 2022, 1:15pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/12 "2022-05-17T13:15:00Z")

</div>

Hello @Yogendra_Pratap_Sing

sorry !!! please mention your index name

`GET _cat/recovery/.ds-ilm-history-5-2022.05.17-000013?v`

please perform a cluster restart which might fix this internal system index to get allocated to the cluster

Keep Posted!!!

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 1:20pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/13 "2022-05-17T13:20:33Z")

</div>

> [@Yogendra\_Pratap\_Sing](#):
>
> ```auto
> [elasticsearch@elasticsearch-master-0 ~]$ curl -XGET http://localhost:9200/_cat/indices?v
> health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
> green open .monitoring-es-7-2022.05.17 0L1HYNueQF245i3UR6Tljg 1 1 14258 8041 16.8mb 9.5mb
> green open filebeat-7.17.3 HmPnIwQBQ9iXuJS3kW3-LA 1 0 148830 0 70.2mb 70.2mb
> green open apm-7.12.1-transaction 6e6rUeowQPOLQJaR8mBeHA 1 0 189045 0 100mb 100mb
> green open apm-7.12.1-metric 56Xipem5RvCl9c7DHt-NHw 1 0 54095 0 10mb 10mb
> green open apm-7.12.1-span MutIu__0Rve4vS72KT6xsg 1 0 215897 0 49.4mb 49.4mb
> green open apm-7.12.1-error azc2O_WqT8CpibocRLBbzA 1 0 41 0 415kb 415kb
> green open .async-search FgI1zzpTTteRBvC3dQsLaw 1 1 2 0 2mb 1mb
> 
> ```

@sudhagar_ramesh could you please know me how can I find index name? I have these indices in my cluster.

```auto
[elasticsearch@elasticsearch-master-0 ~]$ curl -XGET http://localhost:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .monitoring-es-7-2022.05.17 0L1HYNueQF245i3UR6Tljg 1 1 14258 8041 16.8mb 9.5mb
green open filebeat-7.17.3 HmPnIwQBQ9iXuJS3kW3-LA 1 0 148830 0 70.2mb 70.2mb
green open apm-7.12.1-transaction 6e6rUeowQPOLQJaR8mBeHA 1 0 189045 0 100mb 100mb
green open apm-7.12.1-metric 56Xipem5RvCl9c7DHt-NHw 1 0 54095 0 10mb 10mb
green open apm-7.12.1-span MutIu__0Rve4vS72KT6xsg 1 0 215897 0 49.4mb 49.4mb
green open apm-7.12.1-error azc2O_WqT8CpibocRLBbzA 1 0 41 0 415kb 415kb
green open .async-search FgI1zzpTTteRBvC3dQsLaw 1 1 2 0 2mb 1mb

```

---

<div class="post-metadata">

**Author:** ![sudhagar\_ramesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhagar_ramesh/32/105673_2.png) [@sudhagar\_ramesh](https://discuss.elastic.co/u/sudhagar_ramesh)\
**Post date:** [May 17, 2022, 1:21pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/14 "2022-05-17T13:21:59Z")

</div>

@Yogendra_Pratap_Sing could you perform a cluster restart

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 1:24pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/15 "2022-05-17T13:24:26Z")

</div>

Yes @sudhagar_ramesh I deleted previous pod and automatically created a new one so I hope this works what you want

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 1:29pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/16 "2022-05-17T13:29:51Z")

</div>

@sudhagar_ramesh still the same result

---

<div class="post-metadata">

**Author:** ![Yogendra\_Pratap\_Sing](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogendra_pratap_sing/32/105783_2.png) [@Yogendra\_Pratap\_Sing](https://discuss.elastic.co/u/Yogendra_Pratap_Sing)\
**Post date:** [May 17, 2022, 2:24pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/17 "2022-05-17T14:24:25Z")

</div>

When I identify the problem by

```auto
curl -XGET localhost:9200/_cluster/allocation/explain?pretty

```

output is

```auto
{
  "index" : ".ds-ilm-history-5-2022.05.17-000013",
  "shard" : 0,
  "primary" : false,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "CLUSTER_RECOVERED",
    "at" : "2022-05-17T10:59:47.050Z",
    "last_allocation_status" : "no_attempt"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "8S3wqTdGRHq7a7XWWHEyzQ",
      "node_name" : "elasticsearch-master-0",
      "transport_address" : "172.31.46.214:9300",
      "node_attributes" : {
        "ml.machine_memory" : "2147483648",
        "ml.max_open_jobs" : "20",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1073741824",
        "transform.node" : "true"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "replica_after_primary_active",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        },
        {
          "decider" : "throttling",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        }
      ]
    },
    {
      "node_id" : "DbnhybKiRjSc4dgBomSjZw",
      "node_name" : "elasticsearch-master-1",
      "transport_address" : "172.31.30.203:9300",
      "node_attributes" : {
        "ml.machine_memory" : "2147483648",
        "ml.max_open_jobs" : "20",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1073741824",
        "transform.node" : "true"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "replica_after_primary_active",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        },
        {
          "decider" : "throttling",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        }
      ]
    },
    {
      "node_id" : "OLvyXR_rTq-H6DqK0OWceQ",
      "node_name" : "elasticsearch-master-2",
      "transport_address" : "172.31.53.196:9300",
      "node_attributes" : {
        "ml.machine_memory" : "2147483648",
        "ml.max_open_jobs" : "20",
        "xpack.installed" : "true",
        "ml.max_jvm_size" : "1073741824",
        "transform.node" : "true"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "replica_after_primary_active",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        },
        {
          "decider" : "throttling",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        }
      ]
    }
  ]
}

```

primary shard for this replica is not yet active

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 14, 2022, 2:24pm UTC](https://discuss.elastic.co/t/getting-elasticsearch-cluster-status-as-red/304939/18 "2022-06-14T14:24:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.