Reindex 2.4 to 5.6

Elastic Stack Elasticsearch
1

We are in the process of re-indexing 2.4 to 5.6. During the re-indexing process the application is able to perform inserts/updates/delete operation to 2.4. So, we are planning to re-index again to copy the missing documents and replicate update/delete operation. What is the re-index option we need to use to achieve this?

2

Reindex runs based on the data that is available when it starts and will not catch subsequent changes or additions. There is no automatic/built-in way to catch up changes made since the last reindex so you need to create a query based on your data, e.g. last modified timestamp, to run a subsequent reindex to catch up.

I suspect it will be very hard to do this without any downtime.

3

How about the following steps?

  1. Stop the apps
  2. Reindex from 2.4 to 5.6.
  3. Point the apps to 5.6 indices by changing the aliases.
  4. Bring up the apps
  5. Reindex again from 2.4 to 5.6 to catch up the inserts/updates/deletes with the following options:

POST _reindex
{
"source": {
"index": "ES2.4Indices"
},
"dest": {
"index": "ES5.6Indices",
"version_type": "external"
}
}

4

I'd recommend reindexing directly to 7.16.
It's a waste of time IMO to reindex to something which is not supported anymore and did not benefit from all the security patches.

You could use the reindex from remote API for this.

2 Likes
5

I have to agree with David, going from one unsupported version to another doesn't make the process easier.

6

@dadoonet @warkolm Thanks for your response. We host PB's of data in Elasticsearch. So, we can't do remote re-indexing due to the volume of data. This is the reason we are planning to upgrade to ES 5.6 at this moment. Once all indices are in 5.6 we are planning to upgrade to 6.8 and eventually 7.x. Can you please let me know if the steps mentioned in the previous message will work?

7

It looks ok, yes.

However once you cross into 6.X and 7.X you will have mapping changes you need to work on.

8

ok. Thank you @warkolm @dadoonet @Christian_Dahlqvist

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.