Zero downtime upgrade

Hi All,

I would like to know about the ways to upgrade or apply new patches in
Elasticsearch with zero downtime.

Thanks,
Ankit Jain

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Clinton wrote a Gist about it: https://gist.github.com/clintongormley/3888120

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 25 juin 2013 à 08:17, Ankit Jain ankitjaincs06@gmail.com a écrit :

Hi All,

I would like to know about the ways to upgrade or apply new patches in Elasticsearch with zero downtime.

Thanks,
Ankit Jain

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

That recipe has the incredibly naive assumption of having a cluster with
only two nodes. Long story short, you can't. You probably can, but it would
require a lot of work. An entire secondary cluster would need to be setup
and populated with data. In the past, I found that even two different
versions on the network can cause issues with multicast zen discovery. If
you are using the TransportClient, they would all have to be reconfigured.
Rolling upgrades was a feature proposed for 1.0, but the community has been
kept in the dark about what it actually will contain.

Cheers,

Ivan

On Mon, Jun 24, 2013 at 11:40 PM, David Pilato david@pilato.fr wrote:

Clinton wrote a Gist about it:
https://gist.github.com/clintongormley/3888120

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 25 juin 2013 à 08:17, Ankit Jain ankitjaincs06@gmail.com a écrit :

Hi All,

I would like to know about the ways to upgrade or apply new patches in
Elasticsearch with zero downtime.

Thanks,
Ankit Jain

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

The critical phase of rolling upgrades is how to suspend ongoing writes and
to redirect new writes into a temporary storage that can be replayed later
automatically. I assume this is the feature that will appear in 1.0.

The steps can be done manually like in Clinton's gist, but it depends also
on the external clients how they write to the cluster. You could force
external clients to drop their connections while migration takes place, so
they have to reconnect later. I doubt this can happen without exceptions
(and maybe errors and write data loss) on client side. By accepting
exceptions and ignoring how clients can resume smoothly, rolling upgrades
are possible. Front-end load balancers can be very helpful for switching to
the new ES cluster in zero downtime.

Jörg

On Tue, Jun 25, 2013 at 4:57 PM, Ivan Brusic ivan@brusic.com wrote:

That recipe has the incredibly naive assumption of having a cluster with
only two nodes. Long story short, you can't. You probably can, but it would
require a lot of work. An entire secondary cluster would need to be setup
and populated with data. In the past, I found that even two different
versions on the network can cause issues with multicast zen discovery. If
you are using the TransportClient, they would all have to be reconfigured.
Rolling upgrades was a feature proposed for 1.0, but the community has been
kept in the dark about what it actually will contain.

Cheers,

Ivan

On Mon, Jun 24, 2013 at 11:40 PM, David Pilato david@pilato.fr wrote:

Clinton wrote a Gist about it:
https://gist.github.com/clintongormley/3888120

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 25 juin 2013 à 08:17, Ankit Jain ankitjaincs06@gmail.com a écrit :

Hi All,

I would like to know about the ways to upgrade or apply new patches in
Elasticsearch with zero downtime.

Thanks,
Ankit Jain

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re-reading my post, I sounded a bit negative. Rolling upgrades is hard
primarily due to the fact that cross-version clusters are not permitted.
The binary protocol is version specific, so you not only have issues
between cluster nodes, but between the client and server if you are using
the Java clients.

On non-trivial clusters, you can "decommission" nodes using the cluster
update API. Sematext has a good blog post about it:
http://www.elasticsearch.org/guide/reference/api/admin-cluster-update-settings/

If using the Java clients, you might need to setup load balancing nodes to
help the transition.

I'm still hoping that rolling upgrades makes it into 1.0

--
Ivan

On Tue, Jun 25, 2013 at 9:37 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

The critical phase of rolling upgrades is how to suspend ongoing writes
and to redirect new writes into a temporary storage that can be replayed
later automatically. I assume this is the feature that will appear in 1.0.

The steps can be done manually like in Clinton's gist, but it depends also
on the external clients how they write to the cluster. You could force
external clients to drop their connections while migration takes place, so
they have to reconnect later. I doubt this can happen without exceptions
(and maybe errors and write data loss) on client side. By accepting
exceptions and ignoring how clients can resume smoothly, rolling upgrades
are possible. Front-end load balancers can be very helpful for switching to
the new ES cluster in zero downtime.

Jörg

On Tue, Jun 25, 2013 at 4:57 PM, Ivan Brusic ivan@brusic.com wrote:

That recipe has the incredibly naive assumption of having a cluster with
only two nodes. Long story short, you can't. You probably can, but it would
require a lot of work. An entire secondary cluster would need to be setup
and populated with data. In the past, I found that even two different
versions on the network can cause issues with multicast zen discovery. If
you are using the TransportClient, they would all have to be reconfigured.
Rolling upgrades was a feature proposed for 1.0, but the community has been
kept in the dark about what it actually will contain.

Cheers,

Ivan

On Mon, Jun 24, 2013 at 11:40 PM, David Pilato david@pilato.fr wrote:

Clinton wrote a Gist about it:
https://gist.github.com/clintongormley/3888120

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 25 juin 2013 à 08:17, Ankit Jain ankitjaincs06@gmail.com a écrit :

Hi All,

I would like to know about the ways to upgrade or apply new patches in
Elasticsearch with zero downtime.

Thanks,
Ankit Jain

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.