Upgrading from very old version of ES with zero down time

Hello,
my ES cluster is still running version 0.20.1. It is time to upgrade. I
know I cannot just use indexes as is and replace the jars by the newest ES.
They are not compatible as far as I understood.
So I need to set up a parallel cluster with the newest ES and some how
transfer all the data with zero down time. The size of the indexes is about
100Gb and the traffic is relatively big, so it could take some time, and
somehow I need to keep the clusters in sync.
Did someone had such experience?
Does someone have any suggestions how to approach this?
I cannot come up with some elegant solution.
Any help is greatly appreciated.
Thank you,
Eugene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/187ebef9-d1ce-436f-8986-c355c853b622%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I would just start another node (or a couple of nodes for security reasons,
mantaining replicas in different machines, etc.) in your cluster with the
new ES version, wait untill data gets copied, turn off every deprecated
(old) version of ES and then restart the nodes you turned-off with after
updating its version.

Alternatively, you can also do something similar to this:

Hope some of this helps.

El jueves, 2 de octubre de 2014 02:28:02 UTC+2, Eugene Strokin escribió:

Hello,
my ES cluster is still running version 0.20.1. It is time to upgrade. I
know I cannot just use indexes as is and replace the jars by the newest ES.
They are not compatible as far as I understood.
So I need to set up a parallel cluster with the newest ES and some how
transfer all the data with zero down time. The size of the indexes is about
100Gb and the traffic is relatively big, so it could take some time, and
somehow I need to keep the clusters in sync.
Did someone had such experience?
Does someone have any suggestions how to approach this?
I cannot come up with some elegant solution.
Any help is greatly appreciated.
Thank you,
Eugene

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bf27a962-87cd-42aa-b47b-1ce9520307a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

If you wish, I can prepare a knapsack backport for 0.20, then you can dump
all your data into an archive file, and reimport the archive into a higher
version.

Jörg

On Thu, Oct 2, 2014 at 2:28 AM, Eugene Strokin eugene@strokin.info wrote:

Hello,
my ES cluster is still running version 0.20.1. It is time to upgrade. I
know I cannot just use indexes as is and replace the jars by the newest ES.
They are not compatible as far as I understood.
So I need to set up a parallel cluster with the newest ES and some how
transfer all the data with zero down time. The size of the indexes is about
100Gb and the traffic is relatively big, so it could take some time, and
somehow I need to keep the clusters in sync.
Did someone had such experience?
Does someone have any suggestions how to approach this?
I cannot come up with some elegant solution.
Any help is greatly appreciated.
Thank you,
Eugene

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/187ebef9-d1ce-436f-8986-c355c853b622%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/187ebef9-d1ce-436f-8986-c355c853b622%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHkKLYL9jGEvgJ0iMk8KBUoZ%3DL0BCpuMMfJu7MkeHBpyA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

FYI I have prepared Knapsack plugin for Elasticsearch 0.20.6

Source code tag:

Link of Plugin ZIP for 0.20.6.2:

http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-knapsack/0.20.6.2/elasticsearch-knapsack-0.20.6.2-plugin.zip

Jörg

On Thu, Oct 2, 2014 at 12:42 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

If you wish, I can prepare a knapsack backport for 0.20, then you can dump
all your data into an archive file, and reimport the archive into a higher
version.

Jörg

On Thu, Oct 2, 2014 at 2:28 AM, Eugene Strokin eugene@strokin.info
wrote:

Hello,
my ES cluster is still running version 0.20.1. It is time to upgrade. I
know I cannot just use indexes as is and replace the jars by the newest ES.
They are not compatible as far as I understood.
So I need to set up a parallel cluster with the newest ES and some how
transfer all the data with zero down time. The size of the indexes is about
100Gb and the traffic is relatively big, so it could take some time, and
somehow I need to keep the clusters in sync.
Did someone had such experience?
Does someone have any suggestions how to approach this?
I cannot come up with some elegant solution.
Any help is greatly appreciated.
Thank you,
Eugene

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/187ebef9-d1ce-436f-8986-c355c853b622%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/187ebef9-d1ce-436f-8986-c355c853b622%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE3-htFL9XGMnOpV2WMZq9fWhj%2B9QjzJOjZty64Hdh0Ww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Your indices should be fine as is. Lucene is guaranteed to be able to read
data from 1 major revision prior. Elasticsearch 0.20 is Lucene 3 and the
latest Elasticsearch is Lucene 4. Because of various bugs at the Lucene
level, you should run an optimize (normally discouraged) to upgrade the
indices within Lucene.

Can you hold off indexing new content during the transition? If you want
zero downtime, it is essential. Also, what type of client are you using?
Although the Java transport client is the most flexible, it can be an issue
for zero downtime upgrades (with versions prior to 1.0).

If you can wait a little longer, I would hold off for Elasticsearch 1.4,
which is currently in beta. I think it would be the biggest release since
1.0.

Cheers,

Ivan

On Thu, Oct 2, 2014 at 7:57 AM, joergprante@gmail.com <joergprante@gmail.com

wrote:

FYI I have prepared Knapsack plugin for Elasticsearch 0.20.6

https://github.com/jprante/elasticsearch-knapsack

Source code tag:

https://github.com/jprante/elasticsearch-knapsack/releases/tag/0.20.6.2

Link of Plugin ZIP for 0.20.6.2:

http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-knapsack/0.20.6.2/elasticsearch-knapsack-0.20.6.2-plugin.zip

Jörg

On Thu, Oct 2, 2014 at 12:42 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

If you wish, I can prepare a knapsack backport for 0.20, then you can
dump all your data into an archive file, and reimport the archive into a
higher version.

Jörg

On Thu, Oct 2, 2014 at 2:28 AM, Eugene Strokin eugene@strokin.info
wrote:

Hello,
my ES cluster is still running version 0.20.1. It is time to upgrade. I
know I cannot just use indexes as is and replace the jars by the newest ES.
They are not compatible as far as I understood.
So I need to set up a parallel cluster with the newest ES and some how
transfer all the data with zero down time. The size of the indexes is about
100Gb and the traffic is relatively big, so it could take some time, and
somehow I need to keep the clusters in sync.
Did someone had such experience?
Does someone have any suggestions how to approach this?
I cannot come up with some elegant solution.
Any help is greatly appreciated.
Thank you,
Eugene

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/187ebef9-d1ce-436f-8986-c355c853b622%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/187ebef9-d1ce-436f-8986-c355c853b622%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE3-htFL9XGMnOpV2WMZq9fWhj%2B9QjzJOjZty64Hdh0Ww%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE3-htFL9XGMnOpV2WMZq9fWhj%2B9QjzJOjZty64Hdh0Ww%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAOXXGjQ%3D%3DKLrsjxY41o-G9NEDBNPfzVw7T9GR_srm5aQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.