# 7.0 Upgrade Assistant

**URL:** <https://discuss.elastic.co/t/7-0-upgrade-assistant/169289>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [February 20, 2019, 8:38pm UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289 "2019-02-20T20:38:42Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [February 20, 2019, 8:38pm UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/1 "2019-02-20T20:38:42Z")

</div>

Hello World!

I'm running Elastic Stack 6.6.1 and per Kibana's `7.0 Upgrade Assistant` (`Management`-\>`7.0 Upgrade Assistant`), I'm seeing following:

- Cluster
  - GCS Repository settings changed ([Documentation](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes-7.0.html#_google_cloud_storage_repository_plugin)) - warning
    - nodes with repository-gcs installed: [XXXX]

- Indicies
  - Index created before 6.0 ([Documentation](https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes-7.0.html)) - critical
    - Index: .security-6
    - Details: this index was created using version: 5.6.4

Unfortunately both `Documentation` links are broken (probably will be up later).

* * *

Did I miss [Upgrading internal indices for 6.x](https://www.elastic.co/guide/en/elastic-stack/6.6/upgrading-elastic-stack.html#upgrade-internal-indices) step or ... ?

Please advise.

---

<div class="post-metadata">

**Author:** ![Yogesh\_Gaikwad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogesh_gaikwad/32/27025_2.png) [@Yogesh\_Gaikwad](https://discuss.elastic.co/u/Yogesh_Gaikwad)\
**Post date:** [February 21, 2019, 6:06am UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/2 "2019-02-21T06:06:04Z")

</div>

Hi @alexus,

From the error message looks like index created before 6.0 have not been upgraded.

I am assuming your current cluster (on version 6.6.1) was upgraded from 5.6.4.  
If you have followed the step then you can check user authentication works with native realm user.  
If login succeeds then the step of an upgrade from 5.6.4 was done, else you will have to  
[Upgrade internal indices from 6.x step](https://www.elastic.co/guide/en/elastic-stack/6.6/upgrading-elastic-stack.html#upgrade-internal-indices) before proceeding with 7.0.  
Hope this helps.

Regards,  
Yogesh Gaikwad

---

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [February 22, 2019, 5:20am UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/3 "2019-02-22T05:20:53Z")

</div>

I thought bellow part was performed:

> POST /\_xpack/migration/upgrade/.security

yet, `Upgrade Assistant` thinks otherwise...

output from above `POST`:

```
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index",
        "index_uuid" : "_na_",
        "index" : ".security"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index",
    "index_uuid" : "_na_",
    "index" : ".security"
  },
  "status" : 404
}

```

I believe `.security` is alias, yet actual index is `.security-6`...

---

<div class="post-metadata">

**Author:** ![Yogesh\_Gaikwad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogesh_gaikwad/32/27025_2.png) [@Yogesh\_Gaikwad](https://discuss.elastic.co/u/Yogesh_Gaikwad)\
**Post date:** [February 22, 2019, 6:20am UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/4 "2019-02-22T06:20:47Z")

</div>

Hi @alexus,

Not sure why it says no security index found,  
could you please check response for `GET /_xpack/migration/assistance` ?  
This should list all the internal indices like `.security`, `.watcher`?

Could you also please attach the logs that you are getting when you do the upgrade?  
This would help us debug further.

Regards,  
Yogesh Gaikwad

---

<div class="post-metadata">

**Author:** ![Yogesh\_Gaikwad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogesh_gaikwad/32/27025_2.png) [@Yogesh\_Gaikwad](https://discuss.elastic.co/u/Yogesh_Gaikwad)\
**Post date:** [February 22, 2019, 9:22am UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/5 "2019-02-22T09:22:42Z")

</div>

Hi @alexus,

I think we have uncovered a bug wherein if the shard allocation is not enabled and you try to do the upgrade internal indices step, you get into a situation where the `.security` index gets deleted.  
The bulk response was not checked and even when there were failures, we continued with the deletion of old index assuming reindex was successful. This is a bug and I can reproduce this.

On your setup, while following [the upgrade guide](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/rolling-upgrades.html) this is what happened:-

1. Disable shard allocation before the upgrade
2. Once you upgrade all the nodes, plugins etc. following the document you need to enable shard allocation (Step no. 8)  
**I think this is what was missed, to enable shard allocation after restart**
3. Then do the [upgrade of internal indices](https://www.elastic.co/guide/en/elastic-stack/6.6/upgrading-elastic-stack.html#upgrade-internal-indices)

If you did not do the step of enabling shard allocation, the upgrade fails due to the bug that we uncovered and the `.security` was deleted.

Thank you for your query and hope this helps. For you to continue on your quest, please enable the cluster shard allocation and then try upgrading the internal indices.

Thanks and Regards,  
Yogesh Gaikwad

---

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [February 22, 2019, 3:02pm UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/6 "2019-02-22T15:02:54Z")

</div>

> GET /\_xpack/migration/assistance

results:

```
{
  "indices" : { }
}

```

and

> POST /\_xpack/migration/upgrade/.security

```
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index",
        "index_uuid" : "_na_",
        "index" : ".security"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index",
    "index_uuid" : "_na_",
    "index" : ".security"
  },
  "status" : 404
}

```

other then above output, elasticsearch master node does not produce any errors...

* * *

`Reenable shard allocation.` is enabled, yet upgrade fails(

---

<div class="post-metadata">

**Author:** ![gbrown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gbrown/32/34482_2.png) [@gbrown](https://discuss.elastic.co/u/gbrown)\
**Post date:** [February 22, 2019, 5:52pm UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/7 "2019-02-22T17:52:46Z")

</div>

Hi,

I think there may be multiple issues at play here, but one thing to note is that the `_xpack/migration` APIs will not be used to prepare for an upgrade to 7.0 - we're still in the process of updating all of our documentation here, but going forward, this will primarily be done through Kibana's Upgrade Assistant rather than the Migration APIs. However, the Kibana functionality to automate reindexing and upgrades of internal indices will not be available until version 6.7 of the Elastic stack.

The broken documentation links are a known issue - the location of that documentation has changed since the release of 6.6, and will be corrected in the next release. In the meantime, you can find the [correct documentation here](https://www.elastic.co/guide/en/elasticsearch/reference/7.0/breaking-changes-7.0.html).

---

<div class="post-metadata">

**Author:** ![Yogesh\_Gaikwad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogesh_gaikwad/32/27025_2.png) [@Yogesh\_Gaikwad](https://discuss.elastic.co/u/Yogesh_Gaikwad)\
**Post date:** [February 23, 2019, 3:09pm UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/8 "2019-02-23T15:09:40Z")

</div>

Hi @alexus,

I think once the `.security` gets deleted I do not think reenabling shard allocation on the upgraded setup would work, here I am assuming that you tried that on your upgraded setup. (As the `.security` index has been deleted there is nothing upgrade)  
You could try from the 5.6.4 backup and retry the upgrade process to follow the steps.

I tried this on my setup while upgrading from 5.6.14 version 6.6.0 version where, before I did the upgrade of `.security` I reenabled shard allocation. If I did this I could see migration is successful but if I do not enable it then the `.security` index is deleted thereby failing any future upgrade tries.

Hope this helps.

Thanks and Regards,  
Yogesh Gaikwad

---

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [March 11, 2019, 3:53pm UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/9 "2019-03-11T15:53:08Z")

</div>

@Yogesh_Gaikwad Even though [exception](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/6?u=alexus) from previous comment states otherwise, `.security` does exists (it is an alias for `.security-6` index), so there is something to upgrade after all ...

---

<div class="post-metadata">

**Author:** ![Yogesh\_Gaikwad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogesh_gaikwad/32/27025_2.png) [@Yogesh\_Gaikwad](https://discuss.elastic.co/u/Yogesh_Gaikwad)\
**Post date:** [March 20, 2019, 12:48am UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/10 "2019-03-20T00:48:51Z")

</div>

Hi @alexus,

I think I missed your comment.  
When you say the `.security-6` index exists, are you able to search on it?  
Could you do a simple search on `.security` index and see if there is anything in the index?  
`GET /.security/_search`

As I mentioned earlier if the upgrade failed (when the shards were not enabled) you may not be able to recover from the current state. I would suggest starting again with from old cluster from backup and to see if following the steps helps you.

Regards,  
Yogesh Gaikwad

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2019, 12:48am UTC](https://discuss.elastic.co/t/7-0-upgrade-assistant/169289/11 "2019-04-17T00:48:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.