Upgrade from 7.17.9 to 8.19.18 to 9.4.3

Elastic Stack Elasticsearch
1

Hi Team,

I am trying to run upgrades on a dockerized ES environment.

With indices created in 7.17.9, I am able to successfuly migrate to 8.19.18, by just bringing up a new container of ES 8.19.18 pointing to the same volume mount containing the data, created by the 7.17.9.

But when migrating from ES to 8.19.18 to 9.4,3 using the same approach, as there are indices created with incompatible version[7.17.9] are existing, migration fails

We can mark those indices as read only, and block writing to the index, that let us migrate to 9.4.3.

But the problem is, after thje migration, with 9.4.3, those indices cannot be made writeable, because of the incompatibility issue.

Using reindexing is not an option for us, because of the following reasons

  1. We do not want the index names changed
  2. The index mappings has source enabled set to false

So,

  1. Could some one advise us what is the right approach here ?
  2. Is there any way, I can mark these indices writeable again ?

Any help on this would be greatly appreciated,

Thanks,
Sundar.

2

Unfortunately reindexing is required to bring this data into a format to which 9.x versions can write. Refer for example to the 7.17 docs which warn against disabling _source in your mappings:

If the _source field isn’t available then a number of features are not supported:
...

  • The ability to reindex from one Elasticsearch index to another, either to change mappings or analysis, or to upgrade an index to a new major version.
3

Hi David Turner - Thanks for your quick response.

Is there any alternate option available ? We may have many indices and with large amount of data in them. Marking them as read only is not a viable solution for us. Because, many of the documents may need further updates, even after the migration. And we cannot have the same document saved in two different indices as well.

4

Unfortunately I do not think you have many options here.

An index created on version 7 will not be writeable in version 9, to solve this you would need to reindex the data, but at the same time you cannot reindex the data because you have _source disabled.

To reindex the data in an alternative way you would need to export your data using some query and then index this in a different new index on version 8.19, once you have the index in version 8.19 you would be able to upgrade to version 9.

Depending on the size of your indices, this would require a lot of work, time and disk space.

If you really want to use the same index name, you could index the exported data in a temporary index (with _source enabled) and after that delete the old index and reindex again in a new index with the old name.

Being honest, without more context on how you use Elasticsearch, I think it would be way more easier to deprecate the old index and start writing into a new one.

5

Yes unfortunately there's no alternative. Elasticsearch 9.x can only write to indices which were created in 9.x or 8.x.