5.x Snapshot compatibility question

Elastic Stack Elasticsearch
1

We are planning upgrade our Elasticsearch to 5.x from 2.3.4, after read snapshot documents on your website, we realized that "A snapshot of an index created in 1.x can not be restored to 5.x."

This will be a problem, we want to make sure all our snapshot is compatible after upgrade

Our situation

  • We took daily snapshots everyday and monthly snapshot once a month. The monthly snapshot should be incrementally taken from daily snapshot, and we delete daily snapshots after monthly snapshot was done.
  • Some of our monthly snapshot (which is in older 1.6.x version) need to resnapped before upgrade to 5.x
  • Some of our monthly snapshot (which is in newer version 2.3.4) was taken after upgrade, but those daily snapshot that monthly is using (incrementally right?) were in old version

Question

  • Is those monthly snapshot in 2.3.4 but were created from 1.6.x daily snapshot are effected for 5.x upgrade?
  • If we take another annually snapshot now (with 2.3.4 version of elasticsearch), will that fix all problem that I have for 1.6.x version old snapshot?
2

If your snapshot contains segments that have been generated with an old version of elasticsearch (1.x), then it will fail.

When you upgraded from 1.x to 2.x, you probably kept some old segments around.
If you snapshot now, those old segments will be backup'ed as well in the snapshot whatever elasticsearch version you are using.

Upgrade API can help to rewrite old segments to a 2.x compatible version. So after an upgrade you can run a new snapshot again which will have only 2.x compatible segments.

But it does not mean that the snapshot will be readable on a 5.x cluster. Read the first paragraph here: Upgrade | Elasticsearch Guide [2.4] | Elastic

What I'd probably do is to create somehow a 5.x cluster, use reindex from remote to send all data to the new cluster. Then perform a snapshot operation from the new cluster.

To answer to your questions:

  • Is those monthly snapshot in 2.3.4 but were created from 1.6.x daily snapshot are effected for 5.x upgrade?

Yes.

  • If we take another annually snapshot now (with 2.3.4 version of elasticsearch), will that fix all problem that I have for 1.6.x version old snapshot?

No if you still have old segments around.
And even if you don't, read again the page I linked to.

My 2 cents.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.