Migration of Elasticsearch indices (58 GB) from 1.x to 2.x taking huge time

[angshuman.ray (NA NA)]

Hi,
I need to migrate indices of 58 GB data from 1.x to 2.x Elasticsearch version but this migration is taking huge time, almost 2 days. It is not feasible for a production environment to shutdown for 2 days for data migration.

Please help me to find-out a way so that migration can be done as well as production system need not to keep shutdown.

Please validate my thinking below.

1. Continue with the migration from 1.x to 2.x with a separate instance of 2.x node.
2. Change the production system Elasticsearch to 2.x version (a separate node than step 1)
3. When the migration got completed then migrated Elsaticsearch 2.x node will be included in the production Elasticsearch cluster.

Regards,
Angshuman

[spinscale (Alexander Reelsen)]

Hey,

can you explain why the migration takes so long? What is the bottleneck? Getting the data out of elasticsearch? how are you reading the data from elasticsearch? please take some more time to properly explain your setup so others can help.

--Alex

[warkolm (Mark Walkom)]

How are you migrating?