Data Migration from 2.3.3 to 8..9.0

Elastic Stack Elasticsearch
1

I have an on-premise elastic cluster running on version 2.3.3. (aka old)
I have another on-premise elastic cluster running on version 8.9.0 (aka new)
I want to migrate data from one index of old cluster to new index of new cluster.I would like to hear some suggestions based on your experience what is the best approach i can take.

2

You need to set up index templates in the new cluster as mappings are likely to have changed significantly. Once that is done you need to perform a reindex from remote from the new cluster. That should as far as I know work even though the version difference is huge. This naturally requires source to be enabled in the old cluster.

3

I tried out the reindex-from-remote option but my source cluster is giving{
"type": "out_of_memory_error",
"reason": "Java heap space"
}its running out of memory but i am not sure why. Any suggestion

4

You probably need to add RAM and heap to your source cluster. What is the specification of it at the moment?

5

I have reindex 10 years of data. the document counts are matching. The only challange i have is to check the data integrity. I can develop a custom tool which can traverse through each document on both the cluster and do the comparision. but is there any easier approach to do this?

6

No, as far as I know I think a custom script is required.