# Kibana Upgrade from 7.12 to 7.13.3 using migrations.enableV2: false

**URL:** <https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808>\
**Category:** Kibana\
**Created:** [September 22, 2021, 6:51am UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808 "2021-09-22T06:51:50Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![qiratnahraf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/qiratnahraf/32/55984_2.png) [@qiratnahraf](https://discuss.elastic.co/u/qiratnahraf)\
**Post date:** [September 22, 2021, 6:51am UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/1 "2021-09-22T06:51:50Z")

</div>

Hi, We upgraded our kibana from 7.12 to 7.13.3 with migrations.enableV2:  
false set in configuration. The kibana instance upgraded successfully  
but Now we are getting following error when trying to save searches or  
rules

> index [.kibana\_2] blocked by: [FORBIDDEN/8/index write (api)];: cluster\_block\_exception

In kibana.log file we are getting the error message

> {"type":"log","@timestamp":"2021-09-22T10:54:33+05:00","tags":["error","plugins","taskManager"],"pid":139807,"message":"Failed to poll for work: ResponseError: Response Error"}.

one more thing, after kibana upgrade, this time new index created as  
".kibana\_task\_manager\_7.13.3\_reindex\_temp" and  
".kibana\_7.13.3\_reindex\_temp". Furthermore the aliases ".kibana" and  
".kibana\_task\_manager" point to ".kibana\_2" and ".kibana\_task\_manager\_2"  
respectively. I am not sure whether the kibana upgrade done normally  
as latest kibana indexes name are different and give impression of  
temporary indexes. please guide me if manually removing the write block  
on .kibana\_2 and .kibana\_task\_manager\_2 work in our case or not.

 ![kibana error](https://us1.discourse-cdn.com/elastic/original/3X/6/5/657925d1a3975a6ecf9b6b37b85d3e309dc2bb2c.png)  
 ![kibana task manager alias](https://us1.discourse-cdn.com/elastic/original/3X/f/8/f83e18d35a25cd54c8c282f9762ab1358cd82aa2.png)  
 ![kibana indeces](https://us1.discourse-cdn.com/elastic/original/3X/1/b/1b619aa5aeb4955c853d55fc52b763e6e9e57b6d.png)

---

<div class="post-metadata">

**Author:** ![tsullivan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tsullivan/32/31077_2.png) [@tsullivan](https://discuss.elastic.co/u/tsullivan)\
**Post date:** [September 24, 2021, 3:03am UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/2 "2021-09-24T03:03:34Z")

</div>

Why did you set `migrations.enableV2: false`? Is this part of an ongoing discussion?

Do you know what caused the cluster block exception?

---

<div class="post-metadata">

**Author:** ![qiratnahraf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/qiratnahraf/32/55984_2.png) [@qiratnahraf](https://discuss.elastic.co/u/qiratnahraf)\
**Post date:** [September 24, 2021, 1:09pm UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/3 "2021-09-24T13:09:16Z")

</div>

Hi ,

We try upgrade initially without setting `migrations.enableV2: false` and it get failed with timeout error. After googling we found a thread which says to get rid off timeout error during kibana upgrade from 7.12 set `migrations.enableV2: false` in configuration. So we kill kibana instance and set `migrations.enableV2: false` in configuration and try again. This time upgrade get success and we get Kibana UI where in health dashboard it shows kibana version 7.13.3. but as in normal upgrade the index created with kibana version, This time newly created index name end with reindex\_temp. and .kibana and .kibana\_task\_manager alias does not point to theses newly created indices.  
The cause of the cluster block exception is that the index .kibana and .kibana\_task\_manager is in write block. When we manually remove this write block the reported error get fix.

Right Now we have one ruining instance of kibana and not able to upgrade and add other instances to cluster, as upgrade with setting `migrations.enableV2: false` does not work. and if we try upgrade with this setting then as soon as we start new kibana instance it put a write block on .kibana and .kibana\_task\_manager and update get failed in timeout. We need expert advise for way forward.

Right now We have Elasticsearch 7.13.3  
kibana 7.13.3 and we have following indices  
kibana\_7.13.3\_reindex\_temp  
.kibana\_task\_manager\_7.13.3\_reindex\_temp

but .kibana and .kibana\_task\_manager point to ".kibana\_2" and ".kibana\_task\_manager\_2"

---

<div class="post-metadata">

**Author:** ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)\
**Post date:** [September 24, 2021, 1:40pm UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/4 "2021-09-24T13:40:47Z")

</div>

Did you find the reason for the timeouts in the initial upgrade ? It could have been the migrations were taking longer than expected and therefore the timeout errors. Increasing the timeout, memory or performance might allow the migrations to complete.

---

<div class="post-metadata">

**Author:** ![qiratnahraf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/qiratnahraf/32/55984_2.png) [@qiratnahraf](https://discuss.elastic.co/u/qiratnahraf)\
**Post date:** [September 25, 2021, 1:05pm UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/5 "2021-09-25T13:05:25Z")

</div>

> [@zx8086](#):
>
> Did you find the reason for the timeouts in the initial upgrade ? It could have been the migrations were taking longer than expected and therefore the timeout errors. Increasing the timeout, memory or performance might allow the migrations to complete.

So what is the way forward now? .  
Just to update we are able to upgrade two more instance of kibana setting `migrations.enableV2: false`. What we observe that each new instance of kibana create a new index with one increment ( for example when we have single instance .kibana alias were pointing to .kibana\_2 index. when new kibana instance started it create .kibana\_3 index and make .kibana alias point to .kibana\_3 index, and same is the case in .kibana\_task\_manager alias). what making us worried is that we are not sure if this upgrade procedure is correct and we will not face any problem in future upgrades.

---

<div class="post-metadata">

**Author:** ![zx8086](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zx8086/32/94917_2.png) [@zx8086](https://discuss.elastic.co/u/zx8086)\
**Post date:** [September 25, 2021, 8:09pm UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/6 "2021-09-25T20:09:07Z")

</div>

I ran into similar problems when upgrading from 7.12.x to 7.13.x due to my fleet agents.

[https://github.com/elastic/kibana/issues/95321](https://github.com/elastic/kibana/issues/95321)

Failed migrations seem to create the additional kibana indices, which happened because of the timeouts. When we provisioned a higher spec node with the same configuration and data and did the upgrade again, the non-occurrence of the timeout and successful migration lead to a complete upgrade.

A debug view of the kibana log might indicate the root caust of the timeouts failing the migration.

---

<div class="post-metadata">

**Author:** ![rudolf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rudolf/32/51200_2.png) [@rudolf](https://discuss.elastic.co/u/rudolf)\
**Post date:** [September 28, 2021, 9:36am UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/7 "2021-09-28T09:36:00Z")

</div>

> [@qiratnahraf](#):
>
> So what is the way forward now? .  
> Just to update we are able to upgrade two more instance of kibana setting `migrations.enableV2: false` . What we observe that each new instance of kibana create a new index with one increment ( for example when we have single instance .kibana alias were pointing to .kibana\_2 index. when new kibana instance started it create .kibana\_3 index and make .kibana alias point to .kibana\_3 index, and same is the case in .kibana\_task\_manager alias). what making us worried is that we are not sure if this upgrade procedure is correct and we will not face any problem in future upgrades.

What you're describing is definitely not the expected behaviour, when two or more Kibana are started on the same version they should use the same index.

Although we don't have enough data to identify the root cause of the migration failing in the first place, the problems you experience after setting `migrations.enableV2: false` seem to stem from the fact that some of your Kibana instances had this configuration setting set, while others didn't. So it sounds like one Kibana instance was using v2 migrations and the other v1.

You have an unusually large amount of documents in your `.kibana` and `.kibana_task_manager` indices so it sounds like you're running into [[7.14] SO migration takes too long on pre-7.13 upgrades with heavy alerting usage · Issue #106308 · elastic/kibana · GitHub](https://github.com/elastic/kibana/issues/106308)

If it's possible, you could upgrade to 7.14.0 which fixes this issue, but otherwise:

1. Shutdown all Kibana instances
2. Follow the workarounds given in [[7.14] SO migration takes too long on pre-7.13 upgrades with heavy alerting usage · Issue #106308 · elastic/kibana · GitHub](https://github.com/elastic/kibana/issues/106308#issuecomment-883642323)
3. Remove `migrations.enableV2: false` from all your Kibana instances
4. Start Kibana

If you experience any further problems, please share the complete Kibana logs to help us identify the root cause.

---

<div class="post-metadata">

**Author:** ![qiratnahraf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/qiratnahraf/32/55984_2.png) [@qiratnahraf](https://discuss.elastic.co/u/qiratnahraf)\
**Post date:** [September 29, 2021, 5:01pm UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/8 "2021-09-29T17:01:49Z")

</div>

Great, so upgrade to 7.14 will work for us. Do we directly jump to 7.15? Or we first need to upgrade to 7.14 to resolve kibana issue first?

Thanks for your help

---

<div class="post-metadata">

**Author:** ![rudolf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rudolf/32/51200_2.png) [@rudolf](https://discuss.elastic.co/u/rudolf)\
**Post date:** [October 1, 2021, 8:52am UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/9 "2021-10-01T08:52:37Z")

</div>

You can go directly to 7.15

In general, you can go from any 6.x, 7.x to any higher 7.x

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 29, 2021, 8:52am UTC](https://discuss.elastic.co/t/kibana-upgrade-from-7-12-to-7-13-3-using-migrations-enablev2-false/284808/10 "2021-10-29T08:52:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
