# \[error\] All shards failed after migration to 6.7.1

**URL:** <https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872>\
**Category:** Elasticsearch\
**Created:** [April 15, 2019, 9:47am UTC](https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872 "2019-04-15T09:47:43Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ludovic9](https://avatars.discourse-cdn.com/v4/letter/l/977dab/32.png) [@Ludovic9](https://discuss.elastic.co/u/Ludovic9)\
**Post date:** [April 15, 2019, 9:47am UTC](https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872/1 "2019-04-15T09:47:44Z")

</div>

Hi  
I have done this morning an upgrade of my elk stack, to 6.3.0 to 6.7.0

here the steps

disable shards allocation  
update elasticserach

when the "active\_shards\_percent\_as\_number" was 50% (yellow state)

update logstash  
update kibana

then I have problems ☹

I have an error when I see the logs of kibana, I was thinking the problem was kibana but I think not.  
In elasticsearch I have this :

> [2019-04-15T11:39:55,167][WARN][r.suppressed] [prod-master-1] path: /.kibana\_task\_manager/\_doc/\_search, params: {ignore\_unavailable=true, index=.kibana\_task\_manager, type=\_doc}  
> org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed

the status is red

> "cluster\_name" : "logging-prod",  
> "status" : "red",  
> "timed\_out" : false,  
> "number\_of\_nodes" : 4,  
> "number\_of\_data\_nodes" : 2,  
> "active\_primary\_shards" : 1507,  
> "active\_shards" : 1507,  
> "relocating\_shards" : 0,  
> "initializing\_shards" : 0,  
> "unassigned\_shards" : 1511,  
> "delayed\_unassigned\_shards" : 0,  
> "number\_of\_pending\_tasks" : 0,  
> "number\_of\_in\_flight\_fetch" : 0,  
> "task\_max\_waiting\_in\_queue\_millis" : 0,  
> "active\_shards\_percent\_as\_number" : 49.93373094764745

I have 2 indices in red : ".kibana\_task\_manager" and ".kibana\_7"

How I can fix this ?

Do I have to enable again shard allocation ?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 15, 2019, 9:52am UTC](https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872/2 "2019-04-15T09:52:11Z")

</div>

> [@Ludovic9](#):
>
> Do I have to enable again shard allocation ?

According to the [upgrade docs](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/restart-upgrade.html) you should still allow primary shard allocation:

```plaintext
PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "primaries"
  }
}

```

However if you have _completely_ disabled shard allocation then these two new indices will have unassigned primaries, leading to a red cluster health. Try and re-enable allocation (at least, set it to `primaries`) to allow these primaries to be assigned.

---

<div class="post-metadata">

**Author:** ![Ludovic9](https://avatars.discourse-cdn.com/v4/letter/l/977dab/32.png) [@Ludovic9](https://discuss.elastic.co/u/Ludovic9)\
**Post date:** [April 15, 2019, 12:54pm UTC](https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872/3 "2019-04-15T12:54:41Z")

</div>

you saved my life 😄

I have done this

> curl -X PUT "127.0.0.1:9200/\_cluster/settings" -H 'Content-Type: application/json' -d'  
> {  
> "persistent": {  
> "cluster.routing.allocation.enable": "primaries"  
> }  
> }  
> '

the status was yellow, then no more error in elasticsearch.  
Then I have restarted Kibana and I had an error with index .kibana-7 so I have deleted it and it's OK now.

what's the difference between none and primary on the allocation shard ?

Thank you for your answer

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [April 15, 2019, 12:58pm UTC](https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872/4 "2019-04-15T12:58:43Z")

</div>

> [@Ludovic9](#):
>
> what's the difference between none and primary on the allocation shard ?

When you create a new index, Elasticsearch has to allocate the shards to nodes before you can write to them. If you set `cluster.routing.allocation.enable: none` then Elasticsearch will never allocate any shards, whereas if you set `cluster.routing.allocation.enable: primaries` then Elasticsearch will allocate primary shards but not replicas. You cannot write to a shard until at least its primary is allocated, so the cluster health reports red when it is not possible to allocate its primary.

It's possible the [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/shards-allocation.html#_shard_allocation_settings) are also useful to you.

---

<div class="post-metadata">

**Author:** ![Ludovic9](https://avatars.discourse-cdn.com/v4/letter/l/977dab/32.png) [@Ludovic9](https://discuss.elastic.co/u/Ludovic9)\
**Post date:** [April 15, 2019, 2:12pm UTC](https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872/5 "2019-04-15T14:12:34Z")

</div>

ok thank you !

I have read the latest doc, it's written "none". Next time, I'll be more aware !!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 13, 2019, 2:12pm UTC](https://discuss.elastic.co/t/error-all-shards-failed-after-migration-to-6-7-1/176872/6 "2019-05-13T14:12:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.