Upgrade from 5.6 to 6.5-OSS without loosing data

Elastic Stack Elasticsearch
1

Dear all,

I have a running Cluster on Centos 7 with the RPM of Elasticsearch 5.6.13 installed and running. Now after all pre-flight checks, I want to upgrade to 6.5.

But I do not find a way to upgrade to the OSS Version of Elasticsearch without losing the data. The OSS RPM is not marked as a possible upgrade for the 5.6.13 RPM of elasticsearch. I would need to remove the old RPM but that what would wipe the data directory.

I run a test upgrade with the non-OSS RPM and even that runs into issues (did not save the mess to check on that).

But what is the proposed upgrade way from 5.6 to 6.5 without losing data or do a completely fresh installation - or re-index work. I had read the upgrade notes and it looks like it would be an easy walk if the data is prepared.

Thank you for helping me out of this situation!
Jan

2

Is it not possible tout do a rolled restart?
Stop node 1
Upgrage node 1
Restart node 1
Stop node 2
Upgrade node 2
Restart node 2
Do this with redistribution of shards OFF.
I'm just Ă  beginner si perhaps It's a Bad Idea...

3

he @Karatyus

I would love todo that, but shifting the data first off a node and than shift it back to the updated node.

Until the Upgrade to 6.x it was always possible to leave the data on the node and no need was given to shift data between the nodes.

What makes me a little more angry is that no clear communication about that is given in the official docs.

4

Are you sure that data will be moving if you follow the instructions?

5

Can you stop shifting the data with the "cadenas" under cerebro to manage your cluster?

6

yes - I tried that.

But the RPM Package (OSS Version) is not marked as possible upgrade for 5.6.13 (default rpm) but as conflict package.

So you are not able to upgrade direct to the OSS package but need to remove elasticsearch first and then install elasticsearch-oss - what will make you loose the data.

The Problem is not that I do not know how to disable data shifting or wipe a node while still having a green cluster (data management is easy if you know how) - but the RPM is not crafted with upgrading users in mind what drives me crazy.

So maybe a nice upgrade way is not described but given.

7

Not sure what the rpm problem is but few things you might want to consider:

  • use the standard version instead of OSS

Or

  • move the data dir to a data-tmp dir
  • same for config dir
  • install the oss rpm version as you wrote (like a clean fresh install)
  • stop elasticsearch
  • move back the data and config dir
  • start elasticsearch

Would that work?

1 Like
8

Sorry @dadoonet for this question, but what Is OSS? I'm beginner, and i try to understand...

9

it looks like not - the elasticsearch-oss rpm is marked as conflict package for elasticsearch.

I would take your words as - we do not have any recommend upgrade path to the elasticsearch-oss package.

10

From the Docs at Install Elasticsearch with RPM | Elasticsearch Guide [6.5] | Elastic

An alternative package which contains only features that are available under the Apache 2.0 license is also available. To install it, use the following baseurl in your elasticsearch.repo file:
baseurl=https://artifacts.elastic.co/packages/oss-6.x/yum

11

Is there any specific reason you aren't using the default packages?

12

local legal restrictions. Before the presence of OSS Packages it was grayisch to use the software (and disable features via startup parameters) but with the given OSS Package I would like to polish that up to follow all local rules.

13

Huh, that's kinda unusual!

14

But a given requirement, I have also opened a issue in Github about that packaging

I just want to have a clear standing - if that is possible, or if that update path is not given.

15

The last place I worked had a requirement to only use OSS software as well. We figured you couldn't build free culture stuff out of proprietary tools. Anyway, you should be able to upgrade from 5.6 to 6.5-OSS. I suspect we need to do some package magic to make rpm happy, but it seems legit.

16

he @nik9000 or @warkolm

are you able to give any estimation when a decision will be available if this is a proposed upgrade or if that will not work?

17

We don't provide timeframes sorry to say.

18

I take that as - we find that valid and will work on this ... as it does not deny that this is a bug.

19

@jalogisch I am not sure if it is completely clear to me what you mean, but if understand it correctly, @dadoonet 's suggestion seems to be pretty valid here.

To upgrade your ES cluster, you still have to restart your ES nodes, so why not to:

  • stop ES
  • move data dir aside (so it will be 100% intact)
  • upgrade/install ES, whatever is available for your case
  • move data dir back
  • start ES

This based on my understanding that your only concern is the data dir will be wiped after upgrade/install and you can resolve everything else, i.e. the conflicts/different repos.

20

He @taraso

I would say yes - in small environments or when you have enough time to move hudge amounts of data and have a downtime during that movement.

My concern is that the OSS package is mentioned as one installation/upgrade option in the documentation but technical it is not possible because of the package meta settings.
In addition, this is a license issue that because of the changed license you need to install a different package that is no clear upgrade.