Stuck upgrading elastic search from 6.x to 7.17

[CapitanRedBeard (Emmett Harper)]

I have two Elasticsearch deployments hosted on the elastic cloud.
My DEV deployment is on v6.8.9 with kibana on v6.6.1 thus it has a "7.0 Upgrade Assistant" in the Management options in kibana.
My PROD deployment is on v6.4.3 with kibana on v6.3.2 and has no "7.0 Upgrade Assistant" option in kibana.

I'm getting end of life on both of these plus I would like to use some of the 7.x and 8.x features so I would like to upgrade both to be the same version. My problem is after looking through the docs I can't for the life of me figure out how to upgrade either environment. For PROD I don't know how get it up to where it has the upgrade assistant feature. And for DEV using the "7.0 Upgrade Assitant" I can't upgrade because I have 2 warning and 1 critical issue under the cluster tab.

Ignoring the 2 warnings ("GCS Repository settings changed" and "TLS v1.0 has been removed from default TLS/SSL protocols"), the critical issue is "Security realm settings structure changed".

The docs says my settings for security realm was configured a certain way and I now need to migrate it to a different schema. My problem is I have no idea where I need to migrate this. If I press "edit deployment" -> "Edit user settings and plugins(0)" there seems to be a place I can append user settings to Elasticsearch.yml but the realm setting isn't here nor do I know what to copy here to get it to work.

Does anyone have some basic tips for me on how to upgrade this? I have no plugins, no special configuration. I setup Elasticsearch about 2 years ago and just want to simply upgrade it to the latest. Whatever migration I need to do I'm willing to try I just don't even know where to do this.

Thanks

[TimV (Tim Vernum)]

You should upgrade to both of your clusters to the latest Elasticsearch 6 release first (that is, 6.8.23).

That will enable the upgrade assistant in your Prod cluster, and also give you the most up to date information about upgrading.

I think the "Security realm settings structure changed" error might be removed when you upgrade. Elastic Cloud should handle the changes for you, and I think newer releases of the upgrade assistant will understand that.

[CapitanRedBeard (Emmett Harper)]

Tim how do I even upgrade to 6.8.23? In the Edit Deployment I'm not seeing any options. I restarted my deployment and that didn't work.

My problem is I don't know how to upgrade at all. I'm trying to follow this guide Rolling upgrades | Elasticsearch Guide [6.8] | Elastic

but I don't understand how the deployment gets actually upgraded. Is there a command I'm missing to upgrade the node when its shut down?

[TimV (Tim Vernum)]

The upgrade option should be on the deployment management page as shown below.

If it's not showing up, you should open a support ticket because it's impossible for us to guess the reasons why you cluster cannot be upgraded without looking at it.

[CapitanRedBeard (Emmett Harper)]

Oh interesting, yeah I don't see that option.

Alright thanks Tim, I'll open up a support ticket to move forward

[CapitanRedBeard (Emmett Harper)]

Just for others wondering I had to fix this by creating a new 8.x deployment and reindexing from my 6.x cluster