# Getting Error "all shards failed" when I go for re-indexing

**URL:** <https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577>\
**Category:** Elasticsearch\
**Created:** [August 11, 2020, 3:05pm UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577 "2020-08-11T15:05:57Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sarwar\_Kori](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarwar_kori/32/73679_2.png) [@Sarwar\_Kori](https://discuss.elastic.co/u/Sarwar_Kori)\
**Post date:** [August 11, 2020, 3:05pm UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/1 "2020-08-11T15:05:58Z")

</div>

{  
"error" : {  
"root\_cause" : ,  
"type" : "search\_phase\_execution\_exception",  
"reason" : "all shards failed",  
"phase" : "query",  
"grouped" : true,  
"failed\_shards" :   
},  
"status" : 503  
}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 11, 2020, 3:07pm UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/2 "2020-08-11T15:07:09Z")

</div>

I think you will need to provide a lot more detail about how you got that error if you want someone to help.

---

<div class="post-metadata">

**Author:** ![Sarwar\_Kori](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarwar_kori/32/73679_2.png) [@Sarwar\_Kori](https://discuss.elastic.co/u/Sarwar_Kori)\
**Post date:** [August 11, 2020, 3:10pm UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/3 "2020-08-11T15:10:24Z")

</div>

I have issue with two shards , it went in INITIALIZING state because of that my cluster become red.  
Now I want to re-index , without loosing data .

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 11, 2020, 3:12pm UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/4 "2020-08-11T15:12:49Z")

</div>

Please provide more details.

Which version are you running?

What is the size and configuration of your cluster?

What is the use case?

How much data do you have in the cluster?

How did you end up where you are now?

---

<div class="post-metadata">

**Author:** ![Sarwar\_Kori](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarwar_kori/32/73679_2.png) [@Sarwar\_Kori](https://discuss.elastic.co/u/Sarwar_Kori)\
**Post date:** [August 11, 2020, 3:13pm UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/5 "2020-08-11T15:13:40Z")

</div>

Version: 7.6.0  
ClusterSize: 1 node  
Data: 1Tb

Config:  
cluster.name: es-cluster-mo4-rs  
node.master: true  
node.data: true

I have ELK Stack , using it for logs management . I have Life Cycle policy which is rotating index on daily basis. Suddenly one index became in Red state and life policy failed to create new index.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 11, 2020, 3:48pm UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/6 "2020-08-11T15:48:54Z")

</div>

What is the full output of the cluster stats API?

---

<div class="post-metadata">

**Author:** ![Sarwar\_Kori](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarwar_kori/32/73679_2.png) [@Sarwar\_Kori](https://discuss.elastic.co/u/Sarwar_Kori)\
**Post date:** [August 12, 2020, 6:38am UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/7 "2020-08-12T06:38:30Z")

</div>

curl -X GET "x.x.x.x:9200/\_cluster/health?pretty"  
{  
"cluster\_name" : "xxxxxxxxxxx",  
"status" : "red",  
"timed\_out" : false,  
"number\_of\_nodes" : 1,  
"number\_of\_data\_nodes" : 1,  
"active\_primary\_shards" : 29,  
"active\_shards" : 29,  
"relocating\_shards" : 0,  
"initializing\_shards" : 2,  
"unassigned\_shards" : 0,  
"delayed\_unassigned\_shards" : 0,  
"number\_of\_pending\_tasks" : 0,  
"number\_of\_in\_flight\_fetch" : 0,  
"task\_max\_waiting\_in\_queue\_millis" : 0,  
"active\_shards\_percent\_as\_number" : 93.54838709677419  
}

---

<div class="post-metadata">

**Author:** ![Sarwar\_Kori](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarwar_kori/32/73679_2.png) [@Sarwar\_Kori](https://discuss.elastic.co/u/Sarwar_Kori)\
**Post date:** [August 12, 2020, 6:46am UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/8 "2020-08-12T06:46:01Z")

</div>

curl -X GET "x.x.x.x:9200/\_cluster/stats?human&pretty&pretty"

{  
"\_nodes" : {  
"total" : 1,  
"successful" : 1,  
"failed" : 0  
},  
"cluster\_name" : "XXXXXXXXXXXXXXXXXXX",  
"cluster\_uuid" : "5qcyUKYKRSmFhvQCzYzqvQ",  
"timestamp" : 1597214529651,  
"status" : "red",  
"indices" : {  
"count" : 29,  
"shards" : {  
"total" : 29,  
"primaries" : 29,  
"replication" : 0.0,  
"index" : {  
"shards" : {  
"min" : 1,  
"max" : 1,  
"avg" : 1.0  
},  
"primaries" : {  
"min" : 1,  
"max" : 1,  
"avg" : 1.0  
},  
"replication" : {  
"min" : 0.0,  
"max" : 0.0,  
"avg" : 0.0  
}  
}  
},  
"docs" : {  
"count" : 183973730,  
"deleted" : 38  
},  
"store" : {  
"size" : "56.6gb",  
"size\_in\_bytes" : 60866644362  
},  
"fielddata" : {  
"memory\_size" : "0b",  
"memory\_size\_in\_bytes" : 0,  
"evictions" : 0  
},  
"query\_cache" : {  
"memory\_size" : "0b",  
"memory\_size\_in\_bytes" : 0,  
"total\_count" : 0,  
"hit\_count" : 0,  
"miss\_count" : 0,  
"cache\_size" : 0,  
"cache\_count" : 0,  
"evictions" : 0  
},  
"completion" : {  
"size" : "0b",  
"size\_in\_bytes" : 0  
},  
"segments" : {  
"count" : 515,  
"memory" : "96.7mb",  
"memory\_in\_bytes" : 101432595,  
"terms\_memory" : "62.1mb",  
"terms\_memory\_in\_bytes" : 65146095,  
"stored\_fields\_memory" : "34.3mb",  
"stored\_fields\_memory\_in\_bytes" : 36015320,  
"term\_vectors\_memory" : "0b",  
"term\_vectors\_memory\_in\_bytes" : 0,  
"norms\_memory" : "8.6kb",  
"norms\_memory\_in\_bytes" : 8896,  
"points\_memory" : "0b",  
"points\_memory\_in\_bytes" : 0,  
"doc\_values\_memory" : "256.1kb",  
"doc\_values\_memory\_in\_bytes" : 262284,  
"index\_writer\_memory" : "0b",  
"index\_writer\_memory\_in\_bytes" : 0,  
"version\_map\_memory" : "0b",  
"version\_map\_memory\_in\_bytes" : 0,  
"fixed\_bit\_set" : "568b",  
"fixed\_bit\_set\_memory\_in\_bytes" : 568,  
"max\_unsafe\_auto\_id\_timestamp" : 1588912675144,  
"file\_sizes" : { }  
}  
},  
"nodes" : {  
"count" : {  
"total" : 1,  
"coordinating\_only" : 0,  
"data" : 1,  
"ingest" : 1,  
"master" : 1,  
"ml" : 1,  
"voting\_only" : 0  
},  
"versions" : [  
"7.6.0"  
],  
"os" : {  
"available\_processors" : 6,  
"allocated\_processors" : 6,  
"names" : [  
{  
"name" : "Linux",  
"count" : 1  
}  
],  
"pretty\_names" : [  
{  
"pretty\_name" : "CentOS Linux 7 (Core)",  
"count" : 1  
}  
],  
"mem" : {  
"total" : "15.5gb",  
"total\_in\_bytes" : 16646995968,  
"free" : "166.1mb",  
"free\_in\_bytes" : 174231552,  
"used" : "15.3gb",  
"used\_in\_bytes" : 16472764416,  
"free\_percent" : 1,  
"used\_percent" : 99  
}  
},  
"process" : {  
"cpu" : {  
"percent" : 0  
},  
"open\_file\_descriptors" : {  
"min" : 1295,  
"max" : 1295,  
"avg" : 1295  
}  
},  
"jvm" : {  
"max\_uptime" : "17.4h",  
"max\_uptime\_in\_millis" : 62843071,  
"versions" : [  
{  
"version" : "13.0.2",  
"vm\_name" : "OpenJDK 64-Bit Server VM",  
"vm\_version" : "13.0.2+8",  
"vm\_vendor" : "AdoptOpenJDK",  
"bundled\_jdk" : true,  
"using\_bundled\_jdk" : true,  
"count" : 1  
}  
],  
"mem" : {  
"heap\_used" : "1.7gb",  
"heap\_used\_in\_bytes" : 1861839712,  
"heap\_max" : "7.9gb",  
"heap\_max\_in\_bytes" : 8537636864  
},  
"threads" : 80  
},  
"fs" : {  
"total" : "1.4tb",  
"total\_in\_bytes" : 1585199681536,  
"free" : "600.1gb",  
"free\_in\_bytes" : 644449427456,  
"available" : "532.6gb",  
"available\_in\_bytes" : 571923664896  
},  
"plugins" : ,  
"network\_types" : {  
"transport\_types" : {  
"security4" : 1  
},  
"http\_types" : {  
"security4" : 1  
}  
},  
"discovery\_types" : {  
"zen" : 1  
},  
"packaging\_types" : [  
{  
"flavor" : "default",  
"type" : "rpm",  
"count" : 1  
}  
],  
"ingest" : {  
"number\_of\_pipelines" : 1,  
"processor\_stats" : {  
"gsub" : {  
"count" : 0,  
"failed" : 0,  
"current" : 0,  
"time" : "0s",  
"time\_in\_millis" : 0  
},  
"script" : {  
"count" : 0,  
"failed" : 0,  
"current" : 0,  
"time" : "0s",  
"time\_in\_millis" : 0  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 9, 2020, 6:46am UTC](https://discuss.elastic.co/t/getting-error-all-shards-failed-when-i-go-for-re-indexing/244577/9 "2020-09-09T06:46:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
