Elastic search dynamic number of replicas from Java API

goncalo_luiz · July 9, 2014, 9:57pm

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm currently
developing.

This PoC models an application that needs durability but not isolability,
so I'm fine with the eventual consistency of reads against the most recent
writes.

As durability is paramount (we can't affort to lose the data unless 100% of
the nodes die) I've been exploring the option of setting every shard to
have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the number or
replicas which triggers a replication throttled replication process.

I would like to have some help on the following steps (I'm running ES in
embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered to
the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of nodes
in the cluster so I can use their count to set the number of replicas (step

?
4 - is it possible to hook a callback to the event of a node joining or
leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match the new
node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step b)
accordingly
d) - At any time a node might leave the cluster and thus I need to lower
the number or replicas to the new node count (I assume that the cluster
would go ahead and proceed to compensate the lost replica by asking an
existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the nodes
die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or bandwidth
as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · July 9, 2014, 11:21pm

You can set replica number at index creation time or by cluster update
settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction
You will get an index with lower replica number

Yes. Quick code example:

 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }

Yes. Use org.elasticsearch.cluster.ClusterStateListener

From my view your idea of better fault tolerance does not make much sense.
The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When setting
replica number to the number of nodes, the cluster can balance search
requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not isolability,
so I'm fine with the eventual consistency of reads against the most recent
writes.

As durability is paramount (we can't affort to lose the data unless 100%
of the nodes die) I've been exploring the option of setting every shard to
have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the number or
replicas which triggers a replication throttled replication process.

I would like to have some help on the following steps (I'm running ES in
embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered to
the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of nodes
in the cluster so I can use their count to set the number of replicas (step

?
4 - is it possible to hook a callback to the event of a node joining or
leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match the
new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step b)
accordingly
d) - At any time a node might leave the cluster and thus I need to lower
the number or replicas to the new node count (I assume that the cluster
would go ahead and proceed to compensate the lost replica by asking an
existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the nodes
die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or bandwidth
as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Glen_Smith · July 10, 2014, 3:35am

"I've been exploring the option of setting every shard to have N replicas
where N is the number of nodes in the cluster."

You would always have an allocated shard. Do you mean N-1 replicas?
That's not much better of an idea, since the loss
of a single node would then leave you with an unallocated shard.

On Wednesday, July 9, 2014 5:57:53 PM UTC-4, goncal...@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not isolability,
so I'm fine with the eventual consistency of reads against the most recent
writes.

As durability is paramount (we can't affort to lose the data unless 100%
of the nodes die) I've been exploring the option of setting every shard to
have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the number or
replicas which triggers a replication throttled replication process.

I would like to have some help on the following steps (I'm running ES in
embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered to
the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of nodes
in the cluster so I can use their count to set the number of replicas (step

?
4 - is it possible to hook a callback to the event of a node joining or
leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match the
new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step b)
accordingly
d) - At any time a node might leave the cluster and thus I need to lower
the number or replicas to the new node count (I assume that the cluster
would go ahead and proceed to compensate the lost replica by asking an
existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the nodes
die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or bandwidth
as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b0c7e6ba-2845-40ea-9c50-6723566538e8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

goncalo_luiz · July 10, 2014, 6:55am

Hi Joe,

Thanks for your reply.
On this thougth:
"
From my view your idea of better fault tolerance does not make much sense.
The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 of
the nodes will have data of a given index (assuming 0 shards to ease the
discussion), ritght? If those three nodes fail simultaneously the 4th will
have no way of grabbing a copy and data will be lost forever. However if nr
of replicas is 3, the 4th would be able to keep serving the requesrs and
eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joergprante@gmail.com" joergprante@gmail.com wrote:

You can set replica number at index creation time or by cluster update
settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much sense.
The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When setting
replica number to the number of nodes, the cluster can balance search
requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not isolability,
so I'm fine with the eventual consistency of reads against the most recent
writes.

As durability is paramount (we can't affort to lose the data unless 100%
of the nodes die) I've been exploring the option of setting every shard to
have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the number
or replicas which triggers a replication throttled replication process.

I would like to have some help on the following steps (I'm running ES in
embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered to
the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of
nodes in the cluster so I can use their count to set the number of replicas
(step 1) ?
4 - is it possible to hook a callback to the event of a node joining or
leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match the
new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step b)
accordingly
d) - At any time a node might leave the cluster and thus I need to lower
the number or replicas to the new node count (I assume that the cluster
would go ahead and proceed to compensate the lost replica by asking an
existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the
nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or bandwidth
as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ivan · July 10, 2014, 4:38pm

Jörg, have you actually implemented your own ClusterStateListener? I never
had much success. Tried using that interface or
even PublishClusterStateAction.NewClusterStateListener, but either I could
not configure successfully the module (the former) or received no events
(the latter). Implemented on the client side, not as a plugin.

Cheers,

Ivan

On Wed, Jul 9, 2014 at 4:21 PM, joergprante@gmail.com <joergprante@gmail.com

wrote:

Yes. Use org.elasticsearch.cluster.ClusterStateListener

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · July 10, 2014, 4:55pm

On the client side, you can't use cluster state listener, it is for nodes
that have access to a local copy of the master cluster state. Clients must
execute an action to ask for cluster state, and with the current transport
request/response cycle, they must poll for new events ...

Jörg

On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic ivan@brusic.com wrote:

Jörg, have you actually implemented your own ClusterStateListener? I
never had much success. Tried using that interface or
even PublishClusterStateAction.NewClusterStateListener, but either I could
not configure successfully the module (the former) or received no events
(the latter). Implemented on the client side, not as a plugin.

Cheers,

Ivan

On Wed, Jul 9, 2014 at 4:21 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Yes. Use org.elasticsearch.cluster.ClusterStateListener

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ivan · July 10, 2014, 5:03pm

That's what I discovered as well. I would love a client-side non-polling
mechanism. Even tried creating a local non-data/non-master node, but it
was not good enough. One day I will implement it as a plugin to ping an
external monitoring API, but that would be an overkill.

Cheers,

Ivan

On Thu, Jul 10, 2014 at 9:55 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

On the client side, you can't use cluster state listener, it is for nodes
that have access to a local copy of the master cluster state. Clients must
execute an action to ask for cluster state, and with the current transport
request/response cycle, they must poll for new events ...

Jörg

On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic ivan@brusic.com wrote:

Jörg, have you actually implemented your own ClusterStateListener? I
never had much success. Tried using that interface or
even PublishClusterStateAction.NewClusterStateListener, but either I could
not configure successfully the module (the former) or received no events
(the latter). Implemented on the client side, not as a plugin.

Cheers,

Ivan

On Wed, Jul 9, 2014 at 4:21 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Yes. Use org.elasticsearch.cluster.ClusterStateListener

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · July 10, 2014, 5:07pm

All I say is that it depends on the probability of the event of three nodes
failing simultaneously, not on the total number of nodes having a replica.
You can even have 5 nodes and the probability of the event of 4 nodes
failing simultaneously, and so on.

As an illustration, suppose you have a data center with two independent
electric circuits and the probability of failure corresponds with power
outage, then it is enough to distribute nodes equally over servers using
the two independent power lines in the racks. If one electric circuit (plus
UPS) fails, half of the nodes go down. With replica level 1, ES cluster
will keep all the data. There is no need to set replica level equal to node
number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:

Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much sense.
The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 of
the nodes will have data of a given index (assuming 0 shards to ease the
discussion), ritght? If those three nodes fail simultaneously the 4th will
have no way of grabbing a copy and data will be lost forever. However if nr
of replicas is 3, the 4th would be able to keep serving the requesrs and
eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joergprante@gmail.com" joergprante@gmail.com
wrote:
You can set replica number at index creation time or by cluster update
settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When setting
replica number to the number of nodes, the cluster can balance search
requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data unless 100%
of the nodes die) I've been exploring the option of setting every shard to
have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the number
or replicas which triggers a replication throttled replication process.

I would like to have some help on the following steps (I'm running ES in
embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered to
the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of
nodes in the cluster so I can use their count to set the number of replicas
(step 1) ?
4 - is it possible to hook a callback to the event of a node joining or
leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match the
new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step b)
accordingly
d) - At any time a node might leave the cluster and thus I need to lower
the number or replicas to the new node count (I assume that the cluster
would go ahead and proceed to compensate the lost replica by asking an
existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the
nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · July 10, 2014, 5:23pm

For this, an async client implementation is needed that does not close
connection after receiving a response, but waits continuously for a
response stream (events), like I tried in the websocket transport plugin

It would be possible to attach a server-side service with
ClusterStateListener to such a websocket channel for subscribing to cluster
state events. Maybe I find time to implement this for demonstration.

Jörg

On Thu, Jul 10, 2014 at 7:03 PM, Ivan Brusic ivan@brusic.com wrote:

That's what I discovered as well. I would love a client-side non-polling
mechanism. Even tried creating a local non-data/non-master node, but it
was not good enough. One day I will implement it as a plugin to ping an
external monitoring API, but that would be an overkill.

Cheers,

Ivan

On Thu, Jul 10, 2014 at 9:55 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

On the client side, you can't use cluster state listener, it is for nodes
that have access to a local copy of the master cluster state. Clients must
execute an action to ask for cluster state, and with the current transport
request/response cycle, they must poll for new events ...

Jörg

On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic ivan@brusic.com wrote:

Jörg, have you actually implemented your own ClusterStateListener? I
never had much success. Tried using that interface or
even PublishClusterStateAction.NewClusterStateListener, but either I could
not configure successfully the module (the former) or received no events
(the latter). Implemented on the client side, not as a plugin.

Cheers,

Ivan

On Wed, Jul 9, 2014 at 4:21 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Yes. Use org.elasticsearch.cluster.ClusterStateListener

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ivan · July 10, 2014, 6:06pm

I have an idea of how to implement myself, I just don't have the bandwidth
to support it. I pretty much am the sole developer/architect/admin for
our search project, so I am always hesitant to add another moving part.
Will continue to poll for now.

--
Ivan

On Thu, Jul 10, 2014 at 10:23 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

For this, an async client implementation is needed that does not close
connection after receiving a response, but waits continuously for a
response stream (events), like I tried in the websocket transport plugin
GitHub - jprante/elasticsearch-transport-websocket: WebSockets for ElasticSearch

It would be possible to attach a server-side service with
ClusterStateListener to such a websocket channel for subscribing to cluster
state events. Maybe I find time to implement this for demonstration.

Jörg

On Thu, Jul 10, 2014 at 7:03 PM, Ivan Brusic ivan@brusic.com wrote:

That's what I discovered as well. I would love a client-side non-polling
mechanism. Even tried creating a local non-data/non-master node, but it
was not good enough. One day I will implement it as a plugin to ping an
external monitoring API, but that would be an overkill.

Cheers,

Ivan

On Thu, Jul 10, 2014 at 9:55 AM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

On the client side, you can't use cluster state listener, it is for
nodes that have access to a local copy of the master cluster state. Clients
must execute an action to ask for cluster state, and with the current
transport request/response cycle, they must poll for new events ...

Jörg

On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic ivan@brusic.com wrote:

Jörg, have you actually implemented your own ClusterStateListener? I
never had much success. Tried using that interface or
even PublishClusterStateAction.NewClusterStateListener, but either I could
not configure successfully the module (the former) or received no events
(the latter). Implemented on the client side, not as a plugin.

Cheers,

Ivan

On Wed, Jul 9, 2014 at 4:21 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Yes. Use org.elasticsearch.cluster.ClusterStateListener

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBrJS%3DDg0jP_AOw21m-wP6qiTZfTAdTTDJ6%3DzP362_0Fg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

goncalo_luiz · July 10, 2014, 7:39pm

I get it know.

I agree that setting the number of replicas is connected to the deployment
reality in each case and it's derived variables and thus there is no one
formula to fit all cases (it would't be a setting in that case).

What I was trying to cover was the theoretical / extreme case where any
node may fail at any time and what is the best way to go to minimize the
chance of losing data. Also, in the case you want to scale down the
installation (pottentially down to one node) without having to worry about
selecting nodes that hold different replicated shards is an example that
can beneffit from such configuration.

I'm however not clear yet on what happens when a node goes down (triggering
extra replication amongst the survivors) and then comes up again. Is the
ongoing replication cancelled and the returning node brought up to date?

Thanks for your valuable input.

G.
On 10 Jul 2014 18:07, "joergprante@gmail.com" joergprante@gmail.com wrote:

All I say is that it depends on the probability of the event of three
nodes failing simultaneously, not on the total number of nodes having a
replica. You can even have 5 nodes and the probability of the event of 4
nodes failing simultaneously, and so on.

As an illustration, suppose you have a data center with two independent
electric circuits and the probability of failure corresponds with power
outage, then it is enough to distribute nodes equally over servers using
the two independent power lines in the racks. If one electric circuit (plus
UPS) fails, half of the nodes go down. With replica level 1, ES cluster
will keep all the data. There is no need to set replica level equal to node
number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 of
the nodes will have data of a given index (assuming 0 shards to ease the
discussion), ritght? If those three nodes fail simultaneously the 4th will
have no way of grabbing a copy and data will be lost forever. However if nr
of replicas is 3, the 4th would be able to keep serving the requesrs and
eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joergprante@gmail.com" joergprante@gmail.com
wrote:
You can set replica number at index creation time or by cluster
update settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When setting
replica number to the number of nodes, the cluster can balance search
requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data unless
100% of the nodes die) I've been exploring the option of setting every
shard to have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the number
or replicas which triggers a replication throttled replication process.

I would like to have some help on the following steps (I'm running ES
in embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered
to the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of
nodes in the cluster so I can use their count to set the number of replicas
(step 1) ?
4 - is it possible to hook a callback to the event of a node joining or
leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match the
new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step
b) accordingly
d) - At any time a node might leave the cluster and thus I need to
lower the number or replicas to the new node count (I assume that the
cluster would go ahead and proceed to compensate the lost replica by asking
an existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the
nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ivan · July 10, 2014, 7:57pm

Sticking to your use case, you might want to use the auto_expand_replicas
setting to "all" [1]: Never used it, but it sounds what you are looking for.

By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in.

[1]

[2]

Cheers,

Ivan

On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:

I get it know.

I agree that setting the number of replicas is connected to the deployment
reality in each case and it's derived variables and thus there is no one
formula to fit all cases (it would't be a setting in that case).

What I was trying to cover was the theoretical / extreme case where any
node may fail at any time and what is the best way to go to minimize the
chance of losing data. Also, in the case you want to scale down the
installation (pottentially down to one node) without having to worry about
selecting nodes that hold different replicated shards is an example that
can beneffit from such configuration.

I'm however not clear yet on what happens when a node goes down
(triggering extra replication amongst the survivors) and then comes up
again. Is the ongoing replication cancelled and the returning node brought
up to date?

Thanks for your valuable input.

G.
On 10 Jul 2014 18:07, "joergprante@gmail.com" joergprante@gmail.com
wrote:
All I say is that it depends on the probability of the event of three
nodes failing simultaneously, not on the total number of nodes having a
replica. You can even have 5 nodes and the probability of the event of 4
nodes failing simultaneously, and so on.

As an illustration, suppose you have a data center with two independent
electric circuits and the probability of failure corresponds with power
outage, then it is enough to distribute nodes equally over servers using
the two independent power lines in the racks. If one electric circuit (plus
UPS) fails, half of the nodes go down. With replica level 1, ES cluster
will keep all the data. There is no need to set replica level equal to node
number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that 3
of the nodes will have data of a given index (assuming 0 shards to ease the
discussion), ritght? If those three nodes fail simultaneously the 4th will
have no way of grabbing a copy and data will be lost forever. However if nr
of replicas is 3, the 4th would be able to keep serving the requesrs and
eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joergprante@gmail.com" joergprante@gmail.com
wrote:
You can set replica number at index creation time or by cluster
update settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When setting
replica number to the number of nodes, the cluster can balance search
requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data unless
100% of the nodes die) I've been exploring the option of setting every
shard to have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the
number or replicas which triggers a replication throttled replication
process.

I would like to have some help on the following steps (I'm running ES
in embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client ?
2 - What happens if a node dies and the number of replicas is lowered
to the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of
nodes in the cluster so I can use their count to set the number of replicas
(step 1) ?
4 - is it possible to hook a callback to the event of a node joining
or leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match
the new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step
b) accordingly
d) - At any time a node might leave the cluster and thus I need to
lower the number or replicas to the new node count (I assume that the
cluster would go ahead and proceed to compensate the lost replica by asking
an existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the
nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · July 10, 2014, 8:48pm

Indeed, auto_expand_replicas "all" triggers an update cluster settings
action each time a node is added.

Still blown by the many settings Elasticsearch provides. Feeling small.
Homework: collecting a gist textfile of all ES 1.2 settings.

Jörg

On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic ivan@brusic.com wrote:

Sticking to your use case, you might want to use the auto_expand_replicas
setting to "all" [1]: Never used it, but it sounds what you are looking for.

By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Cheers,

Ivan

On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
I get it know.

I agree that setting the number of replicas is connected to the
deployment reality in each case and it's derived variables and thus there
is no one formula to fit all cases (it would't be a setting in that case).

What I was trying to cover was the theoretical / extreme case where any
node may fail at any time and what is the best way to go to minimize the
chance of losing data. Also, in the case you want to scale down the
installation (pottentially down to one node) without having to worry about
selecting nodes that hold different replicated shards is an example that
can beneffit from such configuration.

I'm however not clear yet on what happens when a node goes down
(triggering extra replication amongst the survivors) and then comes up
again. Is the ongoing replication cancelled and the returning node brought
up to date?

Thanks for your valuable input.

G.
On 10 Jul 2014 18:07, "joergprante@gmail.com" joergprante@gmail.com
wrote:
All I say is that it depends on the probability of the event of three
nodes failing simultaneously, not on the total number of nodes having a
replica. You can even have 5 nodes and the probability of the event of 4
nodes failing simultaneously, and so on.

As an illustration, suppose you have a data center with two independent
electric circuits and the probability of failure corresponds with power
outage, then it is enough to distribute nodes equally over servers using
the two independent power lines in the racks. If one electric circuit (plus
UPS) fails, half of the nodes go down. With replica level 1, ES cluster
will keep all the data. There is no need to set replica level equal to node
number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that 3
of the nodes will have data of a given index (assuming 0 shards to ease the
discussion), ritght? If those three nodes fail simultaneously the 4th will
have no way of grabbing a copy and data will be lost forever. However if nr
of replicas is 3, the 4th would be able to keep serving the requesrs and
eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joergprante@gmail.com" joergprante@gmail.com
wrote:
You can set replica number at index creation time or by cluster
update settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When
setting replica number to the number of nodes, the cluster can balance
search requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data unless
100% of the nodes die) I've been exploring the option of setting every
shard to have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the
number or replicas which triggers a replication throttled replication
process.

I would like to have some help on the following steps (I'm running ES
in embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java client
?
2 - What happens if a node dies and the number of replicas is lowered
to the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of
nodes in the cluster so I can use their count to set the number of replicas
(step 1) ?
4 - is it possible to hook a callback to the event of a node joining
or leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match
the new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute step
b) accordingly
d) - At any time a node might leave the cluster and thus I need to
lower the number or replicas to the new node count (I assume that the
cluster would go ahead and proceed to compensate the lost replica by asking
an existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the
nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ivan · July 10, 2014, 9:11pm

It's only been around for 3.5 years:
A setting to auto expand the number of replicas of an index (based on data nodes) · Issue #623 · elastic/elasticsearch · GitHub

I should clarify part of my previous statement.

"By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in."

What I meant to say is that an ongoing recovery is never cancelled once it
has commenced, no matter what settings. By default, recovery happens
immediately, but can be changed with the gateway settings.

--
Ivan

On Thu, Jul 10, 2014 at 1:48 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Indeed, auto_expand_replicas "all" triggers an update cluster settings
action each time a node is added.

Still blown by the many settings Elasticsearch provides. Feeling small.
Homework: collecting a gist textfile of all ES 1.2 settings.

Jörg

On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic ivan@brusic.com wrote:
Sticking to your use case, you might want to use the auto_expand_replicas
setting to "all" [1]: Never used it, but it sounds what you are looking for.

By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Cheers,

Ivan

On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
I get it know.

I agree that setting the number of replicas is connected to the
deployment reality in each case and it's derived variables and thus there
is no one formula to fit all cases (it would't be a setting in that case).

What I was trying to cover was the theoretical / extreme case where any
node may fail at any time and what is the best way to go to minimize the
chance of losing data. Also, in the case you want to scale down the
installation (pottentially down to one node) without having to worry about
selecting nodes that hold different replicated shards is an example that
can beneffit from such configuration.

I'm however not clear yet on what happens when a node goes down
(triggering extra replication amongst the survivors) and then comes up
again. Is the ongoing replication cancelled and the returning node brought
up to date?

Thanks for your valuable input.

G.
On 10 Jul 2014 18:07, "joergprante@gmail.com" joergprante@gmail.com
wrote:
All I say is that it depends on the probability of the event of three
nodes failing simultaneously, not on the total number of nodes having a
replica. You can even have 5 nodes and the probability of the event of 4
nodes failing simultaneously, and so on.

As an illustration, suppose you have a data center with two independent
electric circuits and the probability of failure corresponds with power
outage, then it is enough to distribute nodes equally over servers using
the two independent power lines in the racks. If one electric circuit (plus
UPS) fails, half of the nodes go down. With replica level 1, ES cluster
will keep all the data. There is no need to set replica level equal to node
number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that 3
of the nodes will have data of a given index (assuming 0 shards to ease the
discussion), ritght? If those three nodes fail simultaneously the 4th will
have no way of grabbing a copy and data will be lost forever. However if nr
of replicas is 3, the 4th would be able to keep serving the requesrs and
eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joergprante@gmail.com" joergprante@gmail.com
wrote:
You can set replica number at index creation time or by cluster
update settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When
setting replica number to the number of nodes, the cluster can balance
search requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data unless
100% of the nodes die) I've been exploring the option of setting every
shard to have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the
number or replicas which triggers a replication throttled replication
process.

I would like to have some help on the following steps (I'm running
ES in embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java
client ?
2 - What happens if a node dies and the number of replicas is
lowered to the number of surviving ones?
3 - Is it possible, from a participating node, to access the list of
nodes in the cluster so I can use their count to set the number of replicas
(step 1) ?
4 - is it possible to hook a callback to the event of a node joining
or leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match
the new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute
step b) accordingly
d) - At any time a node might leave the cluster and thus I need to
lower the number or replicas to the new node count (I assume that the
cluster would go ahead and proceed to compensate the lost replica by asking
an existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of the
nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

goncalo_luiz · July 11, 2014, 6:00am

Hi Ivan,

Does this mean that if a note comes back and a replication is underway
we'll end up with a node holding 2 replicas and 1 node holding node ?

Scenario:

Node A - Replica 2
Node B - Replica 3
Node C - Replica 1

If node A dies and Node B get's Replica 2, as soon as node A (or a
replacement) is brought up, is the final configuration likely to be

Node A (or replcament) - No replicas
Node B .- Replica 3 and 2
Node C - Replica 1

or is there a re-balance that takes place ?

Thanks,
Gonçalo

Gonçalo Luiz

On 10 July 2014 22:11, Ivan Brusic ivan@brusic.com wrote:

It's only been around for 3.5 years:
A setting to auto expand the number of replicas of an index (based on data nodes) · Issue #623 · elastic/elasticsearch · GitHub

I should clarify part of my previous statement.

"By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in."

What I meant to say is that an ongoing recovery is never cancelled once it
has commenced, no matter what settings. By default, recovery happens
immediately, but can be changed with the gateway settings.

--
Ivan

On Thu, Jul 10, 2014 at 1:48 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:
Indeed, auto_expand_replicas "all" triggers an update cluster settings
action each time a node is added.

Still blown by the many settings Elasticsearch provides. Feeling small.
Homework: collecting a gist textfile of all ES 1.2 settings.

Jörg

On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic ivan@brusic.com wrote:
Sticking to your use case, you might want to use
the auto_expand_replicas setting to "all" [1]: Never used it, but it sounds
what you are looking for.

By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Cheers,

Ivan

On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
I get it know.

I agree that setting the number of replicas is connected to the
deployment reality in each case and it's derived variables and thus there
is no one formula to fit all cases (it would't be a setting in that case).

What I was trying to cover was the theoretical / extreme case where any
node may fail at any time and what is the best way to go to minimize the
chance of losing data. Also, in the case you want to scale down the
installation (pottentially down to one node) without having to worry about
selecting nodes that hold different replicated shards is an example that
can beneffit from such configuration.

I'm however not clear yet on what happens when a node goes down
(triggering extra replication amongst the survivors) and then comes up
again. Is the ongoing replication cancelled and the returning node brought
up to date?

Thanks for your valuable input.

G.
On 10 Jul 2014 18:07, "joergprante@gmail.com" joergprante@gmail.com
wrote:
All I say is that it depends on the probability of the event of three
nodes failing simultaneously, not on the total number of nodes having a
replica. You can even have 5 nodes and the probability of the event of 4
nodes failing simultaneously, and so on.

As an illustration, suppose you have a data center with two
independent electric circuits and the probability of failure corresponds
with power outage, then it is enough to distribute nodes equally over
servers using the two independent power lines in the racks. If one electric
circuit (plus UPS) fails, half of the nodes go down. With replica level 1,
ES cluster will keep all the data. There is no need to set replica level
equal to node number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz goncalo.luiz@gmail.com
wrote:
Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that
3 of the nodes will have data of a given index (assuming 0 shards to ease
the discussion), ritght? If those three nodes fail simultaneously the 4th
will have no way of grabbing a copy and data will be lost forever. However
if nr of replicas is 3, the 4th would be able to keep serving the requesrs
and eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joergprante@gmail.com" joergprante@gmail.com
wrote:
You can set replica number at index creation time or by cluster
update settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When
setting replica number to the number of nodes, the cluster can balance
search requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncalo.luiz@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data unless
100% of the nodes die) I've been exploring the option of setting every
shard to have N replicas where N is the number of nodes in the cluster.

From what I've read so far it is possible to dynamically set the
number or replicas which triggers a replication throttled replication
process.

I would like to have some help on the following steps (I'm running
ES in embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java
client ?
2 - What happens if a node dies and the number of replicas is
lowered to the number of surviving ones?
3 - Is it possible, from a participating node, to access the list
of nodes in the cluster so I can use their count to set the number of
replicas (step 1) ?
4 - is it possible to hook a callback to the event of a node
joining or leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to match
the new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute
step b) accordingly
d) - At any time a node might leave the cluster and thus I need to
lower the number or replicas to the new node count (I assume that the
cluster would go ahead and proceed to compensate the lost replica by asking
an existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of
the nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMrm09rNQGaaMLE0iOD3NbFWNo7FtmzLh-JswwcKr3Ggp0ztfg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Glen_Smith · July 11, 2014, 11:11am

Hi Goncalo,

I think it's important that you understand: multiple copies of a shard will
never be located on the same node.
Not two replicas, and not the primary and one replica.
To witness this, run a server on your local machine, and create an index
with the defaults - 5 shards, one replica.
You will see that your cluster is "yellow", and has 5 unallocated shards.

How that helps create a better mental picture of shard allocation.

On Friday, July 11, 2014 2:00:47 AM UTC-4, Gonçalo Luiz wrote:

Hi Ivan,

Does this mean that if a note comes back and a replication is underway
we'll end up with a node holding 2 replicas and 1 node holding node ?

Scenario:

Node A - Replica 2
Node B - Replica 3
Node C - Replica 1

If node A dies and Node B get's Replica 2, as soon as node A (or a
replacement) is brought up, is the final configuration likely to be

Node A (or replcament) - No replicas
Node B .- Replica 3 and 2
Node C - Replica 1

or is there a re-balance that takes place ?

Thanks,
Gonçalo

Gonçalo Luiz

On 10 July 2014 22:11, Ivan Brusic <iv...@brusic.com <javascript:>> wrote:
It's only been around for 3.5 years:
A setting to auto expand the number of replicas of an index (based on data nodes) · Issue #623 · elastic/elasticsearch · GitHub

I should clarify part of my previous statement.

"By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in."

What I meant to say is that an ongoing recovery is never cancelled once
it has commenced, no matter what settings. By default, recovery happens
immediately, but can be changed with the gateway settings.

--
Ivan

On Thu, Jul 10, 2014 at 1:48 PM, joerg...@gmail.com <javascript:> <
joerg...@gmail.com <javascript:>> wrote:
Indeed, auto_expand_replicas "all" triggers an update cluster settings
action each time a node is added.

Still blown by the many settings Elasticsearch provides. Feeling small.
Homework: collecting a gist textfile of all ES 1.2 settings.

Jörg

On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic <iv...@brusic.com
<javascript:>> wrote:
Sticking to your use case, you might want to use
the auto_expand_replicas setting to "all" [1]: Never used it, but it sounds
what you are looking for.

By default, the ongoing recovery is not cancelled when the missing node
rejoins the cluster. You can change the gateway settings [2] to control
when recovery kicks in.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Cheers,

Ivan

On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz <goncal...@gmail.com
<javascript:>> wrote:
I get it know.

I agree that setting the number of replicas is connected to the
deployment reality in each case and it's derived variables and thus there
is no one formula to fit all cases (it would't be a setting in that case).

What I was trying to cover was the theoretical / extreme case where
any node may fail at any time and what is the best way to go to minimize
the chance of losing data. Also, in the case you want to scale down the
installation (pottentially down to one node) without having to worry about
selecting nodes that hold different replicated shards is an example that
can beneffit from such configuration.

I'm however not clear yet on what happens when a node goes down
(triggering extra replication amongst the survivors) and then comes up
again. Is the ongoing replication cancelled and the returning node brought
up to date?

Thanks for your valuable input.

G.
On 10 Jul 2014 18:07, "joerg...@gmail.com <javascript:>" <
joerg...@gmail.com <javascript:>> wrote:
All I say is that it depends on the probability of the event of three
nodes failing simultaneously, not on the total number of nodes having a
replica. You can even have 5 nodes and the probability of the event of 4
nodes failing simultaneously, and so on.

As an illustration, suppose you have a data center with two
independent electric circuits and the probability of failure corresponds
with power outage, then it is enough to distribute nodes equally over
servers using the two independent power lines in the racks. If one electric
circuit (plus UPS) fails, half of the nodes go down. With replica level 1,
ES cluster will keep all the data. There is no need to set replica level
equal to node number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz <goncal...@gmail.com
<javascript:>> wrote:
Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means that
3 of the nodes will have data of a given index (assuming 0 shards to ease
the discussion), ritght? If those three nodes fail simultaneously the 4th
will have no way of grabbing a copy and data will be lost forever. However
if nr of replicas is 3, the 4th would be able to keep serving the requesrs
and eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joerg...@gmail.com <javascript:>" <
joerg...@gmail.com <javascript:>> wrote:
You can set replica number at index creation time or by cluster
update settings
action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When
setting replica number to the number of nodes, the cluster can balance
search requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, <goncal...@gmail.com <javascript:>

wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data
unless 100% of the nodes die) I've been exploring the option of setting
every shard to have N replicas where N is the number of nodes in the
cluster.

From what I've read so far it is possible to dynamically set the
number or replicas which triggers a replication throttled replication
process.

I would like to have some help on the following steps (I'm running
ES in embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java
client ?
2 - What happens if a node dies and the number of replicas is
lowered to the number of surviving ones?
3 - Is it possible, from a participating node, to access the list
of nodes in the cluster so I can use their count to set the number of
replicas (step 1) ?
4 - is it possible to hook a callback to the event of a node
joining or leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to
match the new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute
step b) accordingly
d) - At any time a node might leave the cluster and thus I need to
lower the number or replicas to the new node count (I assume that the
cluster would go ahead and proceed to compensate the lost replica by asking
an existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of
the nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7a3b23e6-7e63-4ca3-9a22-4519353a5d2e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

goncalo_luiz · July 11, 2014, 6:36pm

Thanks for the clear and simple explanation.

However, will the cluster (with auto expand replicas) ever go green if it
has been grown from 2 to 3 (triggering replicas to grow to to) and then
downsized to two nodes again? In other words, do the auto grow replicas
setting work both ways or just upwards?

Thanks again.

G.
On 11 Jul 2014 12:11, "Glen Smith" glen@smithsrock.com wrote:

Hi Goncalo,

I think it's important that you understand: multiple copies of a shard
will never be located on the same node.
Not two replicas, and not the primary and one replica.
To witness this, run a server on your local machine, and create an index
with the defaults - 5 shards, one replica.
You will see that your cluster is "yellow", and has 5 unallocated shards.

How that helps create a better mental picture of shard allocation.

On Friday, July 11, 2014 2:00:47 AM UTC-4, Gonçalo Luiz wrote:
Hi Ivan,

Does this mean that if a note comes back and a replication is underway
we'll end up with a node holding 2 replicas and 1 node holding node ?

Scenario:

Node A - Replica 2
Node B - Replica 3
Node C - Replica 1

If node A dies and Node B get's Replica 2, as soon as node A (or a
replacement) is brought up, is the final configuration likely to be

Node A (or replcament) - No replicas
Node B .- Replica 3 and 2
Node C - Replica 1

or is there a re-balance that takes place ?

Thanks,
Gonçalo

Gonçalo Luiz

On 10 July 2014 22:11, Ivan Brusic iv...@brusic.com wrote:
It's only been around for 3.5 years: https://github.com/
elasticsearch/elasticsearch/issues/623

I should clarify part of my previous statement.

"By default, the ongoing recovery is not cancelled when the missing
node rejoins the cluster. You can change the gateway settings [2] to
control when recovery kicks in."

What I meant to say is that an ongoing recovery is never cancelled once
it has commenced, no matter what settings. By default, recovery happens
immediately, but can be changed with the gateway settings.

--
Ivan

On Thu, Jul 10, 2014 at 1:48 PM, joerg...@gmail.com joerg...@gmail.com
wrote:
Indeed, auto_expand_replicas "all" triggers an update cluster
settings action each time a node is added.

Still blown by the many settings Elasticsearch provides. Feeling small.
Homework: collecting a gist textfile of all ES 1.2 settings.

Jörg

On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic iv...@brusic.com wrote:
Sticking to your use case, you might want to use
the auto_expand_replicas setting to "all" [1]: Never used it, but it sounds
what you are looking for.

By default, the ongoing recovery is not cancelled when the missing
node rejoins the cluster. You can change the gateway settings [2] to
control when recovery kicks in.

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/indices-update-settings.html
[2] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/modules-gateway.html

Cheers,

Ivan

On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz goncal...@gmail.com
wrote:
I get it know.

I agree that setting the number of replicas is connected to the
deployment reality in each case and it's derived variables and thus there
is no one formula to fit all cases (it would't be a setting in that case).

What I was trying to cover was the theoretical / extreme case where
any node may fail at any time and what is the best way to go to minimize
the chance of losing data. Also, in the case you want to scale down the
installation (pottentially down to one node) without having to worry about
selecting nodes that hold different replicated shards is an example that
can beneffit from such configuration.

I'm however not clear yet on what happens when a node goes down
(triggering extra replication amongst the survivors) and then comes up
again. Is the ongoing replication cancelled and the returning node brought
up to date?

Thanks for your valuable input.

G.
On 10 Jul 2014 18:07, "joerg...@gmail.com" joerg...@gmail.com
wrote:
All I say is that it depends on the probability of the event of
three nodes failing simultaneously, not on the total number of nodes having
a replica. You can even have 5 nodes and the probability of the event of 4
nodes failing simultaneously, and so on.

As an illustration, suppose you have a data center with two
independent electric circuits and the probability of failure corresponds
with power outage, then it is enough to distribute nodes equally over
servers using the two independent power lines in the racks. If one electric
circuit (plus UPS) fails, half of the nodes go down. With replica level 1,
ES cluster will keep all the data. There is no need to set replica level
equal to node number.

Jörg

On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz goncal...@gmail.com
wrote:
Hi Joe,

Thanks for your reply.
On this thougth:

"
From my view your idea of better fault tolerance does not make much
sense. The replica number is a statistical entity that is related to the
probability of faults. The higher the replica, the higher the probability
of surviving faults. There is no correlation to the total number of nodes
in a cluster to ensure better fault tolerance. The fault tolerance depends
on the probability of a node failure."

I'm not getting it. If we have 4 nodes with 2 replicas it means
that 3 of the nodes will have data of a given index (assuming 0 shards to
ease the discussion), ritght? If those three nodes fail simultaneously the
4th will have no way of grabbing a copy and data will be lost forever.
However if nr of replicas is 3, the 4th would be able to keep serving the
requesrs and eventually handover a copy to a new node joining the cluster.
How does this not help fault tolerance? I'm I missing something?

Thanks,
G.
On 10 Jul 2014 00:21, "joerg...@gmail.com" joerg...@gmail.com
wrote:
You can set replica number at index creation time or by cluster
update settings action org.elasticsearch.
action.admin.cluster.settings.ClusterUpdateSettingsAction

You will get an index with lower replica number
Yes. Quick code example:
 ClusterState clusterState = clusterService.state();
 // find number of data nodes
 int numberOfDataNodes = 0;
 for (DiscoveryNode node : clusterState.getNodes()) {
     if (node.isDataNode()) {
         numberOfDataNodes++;
     }
 }
Yes. Use org.elasticsearch.cluster.ClusterStateListener
From my view your idea of better fault tolerance does not make
much sense. The replica number is a statistical entity that is related to
the probability of faults. The higher the replica, the higher the
probability of surviving faults. There is no correlation to the total
number of nodes in a cluster to ensure better fault tolerance. The fault
tolerance depends on the probability of a node failure.

From the viewpoint of balancing load, it makes much sense. When
setting replica number to the number of nodes, the cluster can balance
search requests to all nodes which is optimal.

Jörg

On Wed, Jul 9, 2014 at 11:57 PM, goncal...@gmail.com wrote:

Hi all,

I'm considering using elasticsearch as a repository for a PoC I'm
currently developing.

This PoC models an application that needs durability but not
isolability, so I'm fine with the eventual consistency of reads against the
most recent writes.

As durability is paramount (we can't affort to lose the data
unless 100% of the nodes die) I've been exploring the option of setting
every shard to have N replicas where N is the number of nodes in the
cluster.

From what I've read so far it is possible to dynamically set the
number or replicas which triggers a replication throttled replication
process.

I would like to have some help on the following steps (I'm
running ES in embedded mode in a Java application):

1 - How can I set the number or replicas using the native Java
client ?
2 - What happens if a node dies and the number of replicas is
lowered to the number of surviving ones?
3 - Is it possible, from a participating node, to access the list
of nodes in the cluster so I can use their count to set the number of
replicas (step 1) ?
4 - is it possible to hook a callback to the event of a node
joining or leaving the cluster ?

I envisioning the following mechanism:

a) - Start with one node, a given number of shards and 1 replica
b)- Each time a node joins I adjust the number or replicas to
match the new node count. In this case, there would be 2 replicas
c) - An arbitrary number of nodes might be added and I'd execute
step b) accordingly
d) - At any time a node might leave the cluster and thus I need
to lower the number or replicas to the new node count (I assume that the
cluster would go ahead and proceed to compensate the lost replica by asking
an existing node to hold 2 replicas instead of one; is this stopped by
lowering the number or replicas?)

The ultimate goal is to make sure no data is loss unless 100% of
the nodes die before a new one can acquire a full replica.

Is this doable? Does this make sense at all ?

For the time being, I'm not worried about lack of disk space or
bandwidth as I'm still in the very early days of the PoC.

Thank you very much for all your work and help.

Gonçalo

--
You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/276418fa-
812f-4af5-94a0-7362f5ba7931%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email
to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/
CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.
gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_
xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%
2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_
S0LWt3tRJHNihWg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_
ROLsSTg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_
GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/hPvVz20v6YY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_
cMkgy6kDSw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMrm09pWmsimVqxRVYf0iSozP_QMrU-_p%3DUODWd9CS4O_Ti5Wg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

jprante · July 11, 2014, 6:41pm

They work both ways.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFLPD9NH%2BTf4aoHCTPdZ4EQ2CnDB75VaeZv%3D%3DV%2BtUUBVA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

ElasticSearch_Users_ · August 22, 2014, 8:08am

Hi Jorg,

Can you please give a server-side or client-side example of using
CLusterStateListener?
Do I have to use a plugin. if so, which module do I register/override?
If not, do I have to use a Node Client (not a TransportClient), and
retrieve the ClusterService somehow and then register?

Thanks
Sandeep

On Thursday, 10 July 2014 22:25:51 UTC+5:30, Jörg Prante wrote:

On the client side, you can't use cluster state listener, it is for nodes
that have access to a local copy of the master cluster state. Clients must
execute an action to ask for cluster state, and with the current transport
request/response cycle, they must poll for new events ...

Jörg

On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic <iv...@brusic.com
<javascript:>> wrote:

Jörg, have you actually implemented your own ClusterStateListener? I
never had much success. Tried using that interface or
even PublishClusterStateAction.NewClusterStateListener, but either I could
not configure successfully the module (the former) or received no events
(the latter). Implemented on the client side, not as a plugin.

Cheers,

Ivan

On Wed, Jul 9, 2014 at 4:21 PM, joerg...@gmail.com <javascript:> <
joerg...@gmail.com <javascript:>> wrote:

Yes. Use org.elasticsearch.cluster.ClusterStateListener

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ivan · August 23, 2014, 3:57pm

As mentioned by Jörg early on in the thread: "On the client side, you can't
use cluster state listener"

I never bothered with a plugin version since the interaction would be too
kludgy.

--
Ivan

On Fri, Aug 22, 2014 at 1:08 AM, 'Sandeep Ramesh Khanzode' via
elasticsearch elasticsearch@googlegroups.com wrote:

Hi Jorg,

Can you please give a server-side or client-side example of using
CLusterStateListener?
Do I have to use a plugin. if so, which module do I register/override?
If not, do I have to use a Node Client (not a TransportClient), and
retrieve the ClusterService somehow and then register?

Thanks
Sandeep

On Thursday, 10 July 2014 22:25:51 UTC+5:30, Jörg Prante wrote:

On the client side, you can't use cluster state listener, it is for nodes
that have access to a local copy of the master cluster state. Clients must
execute an action to ask for cluster state, and with the current transport
request/response cycle, they must poll for new events ...

Jörg

On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic iv...@brusic.com wrote:

Jörg, have you actually implemented your own ClusterStateListener? I
never had much success. Tried using that interface or even
PublishClusterStateAction.NewClusterStateListener, but either I could
not configure successfully the module (the former) or received no events
(the latter). Implemented on the client side, not as a plugin.

Cheers,

Ivan

On Wed, Jul 9, 2014 at 4:21 PM, joerg...@gmail.com joerg...@gmail.com
wrote:

Yes. Use org.elasticsearch.cluster.ClusterStateListener

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-
sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDo1uqMJPjqcx7T4DKiCERHVthEqWEVpwW6jUcKn5x_3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.