Reindex or Clone index (version 1.4)

[cehdmoy (Roberto)]

Hi all, this is my user case

My team has built a solution to index from scratch around 30.000 documents into ES. But the whole process is taking around 2 hours. Not because of ES, it's because to retrieve the data needed to recreate all files from scratch takes too long.

For this reason, we wanna change the strategy, and not recreate all index. Instead, we want to just update some files (maybe 40% of them, defines for business rules), and take metrics to compare.

The new strategy looks like this:

We wanna create two indexes, let call them index_A and index_B, both cloned from production index.

Then for an update by a batch process, update both indexes and choose index_A to production.

Then when running again the batch process it will be executed against index_B, then we gonna flip the indexes and finally apply same updates to index_A for consistency.

But to start this strategy we need to "clone" the production index. We don't' want to create from scratch indexes over an over again.

Here is my question: How could we clone or copy or reindex the index, please notice that reindex API isn't defined for V1.4 according to my understanding. Any suggestions or workaround to archive that.

Thanks in advance.

[Christian_Dahlqvist]

Why are you using version 1.4 if you are developing a new solution??

[cehdmoy (Roberto)]

That's a good question. It's because the indexing process is just a part of the whole process and services, and there is a lot of legacy code in there that is implemented using this version of ES. So update the version isn't trivial. I would love just update it but doesn't look like an option by now sadly.

Regards.

[Christian_Dahlqvist]

I have “cloned” indices using snapshot and restore before, but do not remember when that was introduced. Version 1.4 is almost 5 years old if I remember correctly so I would warmly recommend upgrading (at least to the last 1.7.x release which should be compatible).