Upgrading Elasticsearch 8.19.2 to 9.1.2

Elastic Search
1

I am having a nightmare upgrading from 8.19.2 to 9.1.2.

Self-host

Air-gap netowrk

7 node cluster

RHEL 8.10

Java 17

  1. Upgrade advisory says I am good.
    1. I have two warnings one for Elasticsearch and one for Kibana.
      1. ES: xpack.monitoring.collection.enabled deprecated
      2. Kibana: ILM policy deprecated.
    2. I remove on node from the cluster. rpm -Uvh and it upgrades fine.
    3. When I go to going the upgrade node back in the cluster I get an error saying various indices from es-7.17 is present[examples: .kabana-7.17.1-001, .security-tokens-7….]. I say various because I get a different message per restart.
    4. Is there a way to flush or delete these old indices or migrate them so they are compadable with es 9.X?
    5. I’ve upgrade and downgrade four times now and I don’t want to keep goign through this cycle.
2

Did you click on the “Migrate System Indices” button in the upgrade assistant? It can be easy to miss, and I think it would cause problems like this.