Migrate from 6.8 to 7.10

Hi folks...

Need to know whats the proper way to migrate & upgrade from our current ELK 6.8 cluster to our new ELK 7.10 cluster.

Should I first do rolling upgrade in our current cluster then rsync to 7.10 cluster?
Or use logstash to move the data to our new cluster?

Im in the process of doing this, what i learned:

There's no "proper" way to do it. Everyone needs a different solution based on a use-case. For me, after backing up my AWS cluster, restoring to 6.x (from 5.x), then trying to do a inplace upgrade, i realized this was not possible due to breaking changes: https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html

If none of these will break your index (it most likely will), you can use whatever you'd like (snapshot 6.x > restore to 7.x - this is the fastest way). However if it will break your index, then you need to figure that out.

For me i just ended up reindexing my entire 2.2 TB index. It took less than a day, and would've been much faster than using the reindex API and other solutions available.

I don't think that's true. You can't open a 5.x index in 7.x, but once you've addressed that (and any other deprecation warnings) an in-place upgrade is the way to go.

To migrate to a different cluster at the same time, as the OP wants, I would recommend restoring a snapshot of the old cluster into the new one.

1 Like

This is true, It might not. However; for me, it did. Specifically this: https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#default-mapping-not-allowed

Maybe it will more likely break if you're going 5 > 7 (like i did) than it is 6 > 7 (what OP is doing). It all depends on how they're setup and what things they're using that were deprecated/removed. It's always best to review the breaking changes before you update as to not run into issues.

But yes, as David and I pointed out, if there are no breaking changes affecting your cluster, then a simple snapshot and restore is the way to go (make sure you figure out/address any breaking changes first).

You can definitely address that in 6.x and then perform an in-place upgrade to 7.x as I described.

That is true. I don't recall saying it wasn't possible nor not an option!

Thanks for all the response guys...
Definitely would try snapshot approach