Cluster state when gateway is down

Elastic Stack Elasticsearch
1

First off, I was thrilled when I tested out what would happen when my
gateway goes away. It appears that the cluster snapshot just fails
that time around and all the transactions pending snapshotting are
kept until the gateway is available again and all is snapshotted and
happy. So, this means you don't need to ensure 100% gateway uptime.

However, it would be awesome if the cluster health could somehow
reflect that the gateway is unavailable. While the gateway is
unavailable, large swaths of data could be lost if a cluster shutdown
occurs.

Any thoughts?

Thanks,
Paul

2

Agreed. Need to think which API would best fit that, the cluster health or
the index status API. I plan to also add index stats API which will probably
include how long and when the last snapshot was taken (and if there is on in
progress "now").

-shay.banon

On Mon, Aug 9, 2010 at 1:14 AM, Paul ppearcy@gmail.com wrote:

First off, I was thrilled when I tested out what would happen when my
gateway goes away. It appears that the cluster snapshot just fails
that time around and all the transactions pending snapshotting are
kept until the gateway is available again and all is snapshotted and
happy. So, this means you don't need to ensure 100% gateway uptime.

However, it would be awesome if the cluster health could somehow
reflect that the gateway is unavailable. While the gateway is
unavailable, large swaths of data could be lost if a cluster shutdown
occurs.

Any thoughts?

Thanks,
Paul

3

Ohh, and to reassure you, yes, this is by design that the cluster can
survive gateway "(long) hiccups".

-shay.banon

On Mon, Aug 9, 2010 at 1:18 AM, Shay Banon shay.banon@elasticsearch.comwrote:

Agreed. Need to think which API would best fit that, the cluster health or
the index status API. I plan to also add index stats API which will probably
include how long and when the last snapshot was taken (and if there is on in
progress "now").

-shay.banon

On Mon, Aug 9, 2010 at 1:14 AM, Paul ppearcy@gmail.com wrote:

First off, I was thrilled when I tested out what would happen when my
gateway goes away. It appears that the cluster snapshot just fails
that time around and all the transactions pending snapshotting are
kept until the gateway is available again and all is snapshotted and
happy. So, this means you don't need to ensure 100% gateway uptime.

However, it would be awesome if the cluster health could somehow
reflect that the gateway is unavailable. While the gateway is
unavailable, large swaths of data could be lost if a cluster shutdown
occurs.

Any thoughts?

Thanks,
Paul

4

Yeah, very cool and impressive stuff :slight_smile:

FYI, I opened this ticket to track:

On Aug 8, 4:19 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Ohh, and to reassure you, yes, this is by design that the cluster can
survive gateway "(long) hiccups".

-shay.banon

On Mon, Aug 9, 2010 at 1:18 AM, Shay Banon shay.ba...@elasticsearch.comwrote:

Agreed. Need to think which API would best fit that, the cluster health or
the index status API. I plan to also add index stats API which will probably
include how long and when the last snapshot was taken (and if there is on in
progress "now").

-shay.banon

On Mon, Aug 9, 2010 at 1:14 AM, Paul ppea...@gmail.com wrote:

First off, I was thrilled when I tested out what would happen when my
gateway goes away. It appears that the cluster snapshot just fails
that time around and all the transactions pending snapshotting are
kept until the gateway is available again and all is snapshotted and
happy. So, this means you don't need to ensure 100% gateway uptime.

However, it would be awesome if the cluster health could somehow
reflect that the gateway is unavailable. While the gateway is
unavailable, large swaths of data could be lost if a cluster shutdown
occurs.

Any thoughts?

Thanks,
Paul