# Elastic search dynamic number of replicas from Java API **URL:** \ **Category:** Elasticsearch\ **Created:** [July 9, 2014, 9:57pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563 "2014-07-09T21:57:53Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![goncalo\_luiz](https://avatars.discourse-cdn.com/v4/letter/g/e99b99/32.png) [@goncalo\_luiz](https://discuss.elastic.co/u/goncalo_luiz)\ **Post date:** [July 9, 2014, 9:57pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/1 "2014-07-09T21:57:53Z")
Hi all, I'm considering using elasticsearch as a repository for a PoC I'm currently developing. This PoC models an application that needs durability but not isolability, so I'm fine with the eventual consistency of reads against the most recent writes. As durability is paramount (we can't affort to lose the data unless 100% of the nodes die) I've been exploring the option of setting every shard to have N replicas where N is the number of nodes in the cluster. From what I've read so far it is possible to dynamically set the number or replicas which triggers a replication throttled replication process. I would like to have some help on the following steps (I'm running ES in embedded mode in a Java application): 1 - How can I set the number or replicas using the native Java client ? 2 - What happens if a node dies and the number of replicas is lowered to the number of surviving ones? 3 - Is it possible, from a participating node, to access the list of nodes in the cluster so I can use their count to set the number of replicas (step 1. ? 4 - is it possible to hook a callback to the event of a node joining or leaving the cluster ? I envisioning the following mechanism: a) - Start with one node, a given number of shards and 1 replica b)- Each time a node joins I adjust the number or replicas to match the new node count. In this case, there would be 2 replicas c) - An arbitrary number of nodes might be added and I'd execute step b) accordingly d) - At any time a node might leave the cluster and thus I need to lower the number or replicas to the new node count (I assume that the cluster would go ahead and proceed to compensate the lost replica by asking an existing node to hold 2 replicas instead of one; is this stopped by lowering the number or replicas?) The ultimate goal is to make sure no data is loss unless 100% of the nodes die before a new one can acquire a full replica. Is this doable? Does this make sense at all ? For the time being, I'm not worried about lack of disk space or bandwidth as I'm still in the very early days of the PoC. Thank you very much for all your work and help. Gonçalo -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 9, 2014, 11:21pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/2 "2014-07-09T23:21:36Z")
1. You can set replica number at index creation time or by cluster update settings action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction 2. You will get an index with lower replica number 🙂 3. Yes. Quick code example: 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener From my view your idea of better fault tolerance does not make much sense. The replica number is a statistical entity that is related to the probability of faults. The higher the replica, the higher the probability of surviving faults. There is no correlation to the total number of nodes in a cluster to ensure better fault tolerance. The fault tolerance depends on the probability of a node failure. From the viewpoint of balancing load, it makes much sense. When setting replica number to the number of nodes, the cluster can balance search requests to all nodes which is optimal. Jörg On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > Hi all, > > I'm considering using elasticsearch as a repository for a PoC I'm > currently developing. > > This PoC models an application that needs durability but not isolability, > so I'm fine with the eventual consistency of reads against the most recent > writes. > > As durability is paramount (we can't affort to lose the data unless 100% > of the nodes die) I've been exploring the option of setting every shard to > have N replicas where N is the number of nodes in the cluster. > > From what I've read so far it is possible to dynamically set the number or > replicas which triggers a replication throttled replication process. > > I would like to have some help on the following steps (I'm running ES in > embedded mode in a Java application): > > 1 - How can I set the number or replicas using the native Java client ? > 2 - What happens if a node dies and the number of replicas is lowered to > the number of surviving ones? > 3 - Is it possible, from a participating node, to access the list of nodes > in the cluster so I can use their count to set the number of replicas (step > > 1. ? > 4 - is it possible to hook a callback to the event of a node joining or > leaving the cluster ? > > I envisioning the following mechanism: > > a) - Start with one node, a given number of shards and 1 replica > b)- Each time a node joins I adjust the number or replicas to match the > new node count. In this case, there would be 2 replicas > c) - An arbitrary number of nodes might be added and I'd execute step b) > accordingly > d) - At any time a node might leave the cluster and thus I need to lower > the number or replicas to the new node count (I assume that the cluster > would go ahead and proceed to compensate the lost replica by asking an > existing node to hold 2 replicas instead of one; is this stopped by > lowering the number or replicas?) > > The ultimate goal is to make sure no data is loss unless 100% of the nodes > die before a new one can acquire a full replica. > > Is this doable? Does this make sense at all ? > > For the time being, I'm not worried about lack of disk space or bandwidth > as I'm still in the very early days of the PoC. > > Thank you very much for all your work and help. > > Gonçalo > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Glen\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glen_smith/32/111656_2.png) [@Glen\_Smith](https://discuss.elastic.co/u/Glen_Smith)\ **Post date:** [July 10, 2014, 3:35am UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/3 "2014-07-10T03:35:30Z")
"I've been exploring the option of setting every shard to have N replicas where N is the number of nodes in the cluster." You would _always_ have an allocated shard. Do you mean N-1 replicas? That's not much better of an idea, since the loss of a single node would then leave you with an unallocated shard. On Wednesday, July 9, 2014 5:57:53 PM UTC-4, [goncal...@gmail.com](mailto:goncal...@gmail.com) wrote: > Hi all, > > I'm considering using elasticsearch as a repository for a PoC I'm > currently developing. > > This PoC models an application that needs durability but not isolability, > so I'm fine with the eventual consistency of reads against the most recent > writes. > > As durability is paramount (we can't affort to lose the data unless 100% > of the nodes die) I've been exploring the option of setting every shard to > have N replicas where N is the number of nodes in the cluster. > > From what I've read so far it is possible to dynamically set the number or > replicas which triggers a replication throttled replication process. > > I would like to have some help on the following steps (I'm running ES in > embedded mode in a Java application): > > 1 - How can I set the number or replicas using the native Java client ? > 2 - What happens if a node dies and the number of replicas is lowered to > the number of surviving ones? > 3 - Is it possible, from a participating node, to access the list of nodes > in the cluster so I can use their count to set the number of replicas (step > > 1. ? > 4 - is it possible to hook a callback to the event of a node joining or > leaving the cluster ? > > I envisioning the following mechanism: > > a) - Start with one node, a given number of shards and 1 replica > b)- Each time a node joins I adjust the number or replicas to match the > new node count. In this case, there would be 2 replicas > c) - An arbitrary number of nodes might be added and I'd execute step b) > accordingly > d) - At any time a node might leave the cluster and thus I need to lower > the number or replicas to the new node count (I assume that the cluster > would go ahead and proceed to compensate the lost replica by asking an > existing node to hold 2 replicas instead of one; is this stopped by > lowering the number or replicas?) > > The ultimate goal is to make sure no data is loss unless 100% of the nodes > die before a new one can acquire a full replica. > > Is this doable? Does this make sense at all ? > > For the time being, I'm not worried about lack of disk space or bandwidth > as I'm still in the very early days of the PoC. > > Thank you very much for all your work and help. > > Gonçalo -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b0c7e6ba-2845-40ea-9c50-6723566538e8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b0c7e6ba-2845-40ea-9c50-6723566538e8%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![goncalo\_luiz](https://avatars.discourse-cdn.com/v4/letter/g/e99b99/32.png) [@goncalo\_luiz](https://discuss.elastic.co/u/goncalo_luiz)\ **Post date:** [July 10, 2014, 6:55am UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/4 "2014-07-10T06:55:24Z")
Hi Joe, Thanks for your reply. On this thougth: " From my view your idea of better fault tolerance does not make much sense. The replica number is a statistical entity that is related to the probability of faults. The higher the replica, the higher the probability of surviving faults. There is no correlation to the total number of nodes in a cluster to ensure better fault tolerance. The fault tolerance depends on the probability of a node failure." I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 of the nodes will have data of a given index (assuming 0 shards to ease the discussion), ritght? If those three nodes fail simultaneously the 4th will have no way of grabbing a copy and data will be lost forever. However if nr of replicas is 3, the 4th would be able to keep serving the requesrs and eventually handover a copy to a new node joining the cluster. How does this not help fault tolerance? I'm I missing something? Thanks, G. On 10 Jul 2014 00:21, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > 1. You can set replica number at index creation time or by cluster update > settings > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > 2. You will get an index with lower replica number 🙂 > > 3. Yes. Quick code example: > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > From my view your idea of better fault tolerance does not make much sense. > The replica number is a statistical entity that is related to the > probability of faults. The higher the replica, the higher the probability > of surviving faults. There is no correlation to the total number of nodes > in a cluster to ensure better fault tolerance. The fault tolerance depends > on the probability of a node failure. > > From the viewpoint of balancing load, it makes much sense. When setting > replica number to the number of nodes, the cluster can balance search > requests to all nodes which is optimal. > > Jörg > > On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > > > Hi all, > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > currently developing. > > > > This PoC models an application that needs durability but not isolability, > > so I'm fine with the eventual consistency of reads against the most recent > > writes. > > > > As durability is paramount (we can't affort to lose the data unless 100% > > of the nodes die) I've been exploring the option of setting every shard to > > have N replicas where N is the number of nodes in the cluster. > > > > From what I've read so far it is possible to dynamically set the number > > or replicas which triggers a replication throttled replication process. > > > > I would like to have some help on the following steps (I'm running ES in > > embedded mode in a Java application): > > > > 1 - How can I set the number or replicas using the native Java client ? > > 2 - What happens if a node dies and the number of replicas is lowered to > > the number of surviving ones? > > 3 - Is it possible, from a participating node, to access the list of > > nodes in the cluster so I can use their count to set the number of replicas > > (step 1) ? > > 4 - is it possible to hook a callback to the event of a node joining or > > leaving the cluster ? > > > > I envisioning the following mechanism: > > > > a) - Start with one node, a given number of shards and 1 replica > > b)- Each time a node joins I adjust the number or replicas to match the > > new node count. In this case, there would be 2 replicas > > c) - An arbitrary number of nodes might be added and I'd execute step b) > > accordingly > > d) - At any time a node might leave the cluster and thus I need to lower > > the number or replicas to the new node count (I assume that the cluster > > would go ahead and proceed to compensate the lost replica by asking an > > existing node to hold 2 replicas instead of one; is this stopped by > > lowering the number or replicas?) > > > > The ultimate goal is to make sure no data is loss unless 100% of the > > nodes die before a new one can acquire a full replica. > > > > Is this doable? Does this make sense at all ? > > > > For the time being, I'm not worried about lack of disk space or bandwidth > > as I'm still in the very early days of the PoC. > > > > Thank you very much for all your work and help. > > > > Gonçalo > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 10, 2014, 4:38pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/5 "2014-07-10T16:38:08Z")
Jörg, have you actually implemented your own ClusterStateListener? I never had much success. Tried using that interface or even PublishClusterStateAction.NewClusterStateListener, but either I could not configure successfully the module (the former) or received no events (the latter). Implemented on the client side, not as a plugin. Cheers, Ivan On Wed, Jul 9, 2014 at 4:21 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<[joergprante@gmail.com](mailto:joergprante@gmail.com) > wrote: > 1. Yes. Use org.elasticsearch.cluster.ClusterStateListener -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 10, 2014, 4:55pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/6 "2014-07-10T16:55:44Z")
On the client side, you can't use cluster state listener, it is for nodes that have access to a local copy of the master cluster state. Clients must execute an action to ask for cluster state, and with the current transport request/response cycle, they must poll for new events ... Jörg On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > Jörg, have you actually implemented your own ClusterStateListener? I > never had much success. Tried using that interface or > even PublishClusterStateAction.NewClusterStateListener, but either I could > not configure successfully the module (the former) or received no events > (the latter). Implemented on the client side, not as a plugin. > > Cheers, > > Ivan > > On Wed, Jul 9, 2014 at 4:21 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > 1. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf\_\_vo0gGh\_53gkXjKA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 10, 2014, 5:03pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/7 "2014-07-10T17:03:05Z")
That's what I discovered as well. I would love a client-side non-polling mechanism. Even tried creating a local non-data/non-master node, but it was not good enough. One day I will implement it as a plugin to ping an external monitoring API, but that would be an overkill. Cheers, Ivan On Thu, Jul 10, 2014 at 9:55 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > On the client side, you can't use cluster state listener, it is for nodes > that have access to a local copy of the master cluster state. Clients must > execute an action to ask for cluster state, and with the current transport > request/response cycle, they must poll for new events ... > > Jörg > > On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > Jörg, have you actually implemented your own ClusterStateListener? I > > never had much success. Tried using that interface or > > even PublishClusterStateAction.NewClusterStateListener, but either I could > > not configure successfully the module (the former) or received no events > > (the latter). Implemented on the client side, not as a plugin. > > > > Cheers, > > > > Ivan > > > > On Wed, Jul 9, 2014 at 4:21 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > > > 1. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf\_\_vo0gGh\_53gkXjKA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf\_\_vo0gGh\_53gkXjKA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 10, 2014, 5:07pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/8 "2014-07-10T17:07:30Z")
All I say is that it depends on the probability of the event of three nodes failing simultaneously, not on the total number of nodes having a replica. You can even have 5 nodes and the probability of the event of 4 nodes failing simultaneously, and so on. As an illustration, suppose you have a data center with two independent electric circuits and the probability of failure corresponds with power outage, then it is enough to distribute nodes equally over servers using the two independent power lines in the racks. If one electric circuit (plus UPS) fails, half of the nodes go down. With replica level 1, ES cluster will keep all the data. There is no need to set replica level equal to node number. Jörg On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > Hi Joe, > > Thanks for your reply. > On this thougth: > > " > From my view your idea of better fault tolerance does not make much sense. > The replica number is a statistical entity that is related to the > probability of faults. The higher the replica, the higher the probability > of surviving faults. There is no correlation to the total number of nodes > in a cluster to ensure better fault tolerance. The fault tolerance depends > on the probability of a node failure." > > I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 of > the nodes will have data of a given index (assuming 0 shards to ease the > discussion), ritght? If those three nodes fail simultaneously the 4th will > have no way of grabbing a copy and data will be lost forever. However if nr > of replicas is 3, the 4th would be able to keep serving the requesrs and > eventually handover a copy to a new node joining the cluster. > How does this not help fault tolerance? I'm I missing something? > > Thanks, > G. > On 10 Jul 2014 00:21, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > wrote: > > > 1. You can set replica number at index creation time or by cluster update > > settings > > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > 2. You will get an index with lower replica number 🙂 > > > > 3. Yes. Quick code example: > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > From my view your idea of better fault tolerance does not make much > > sense. The replica number is a statistical entity that is related to the > > probability of faults. The higher the replica, the higher the probability > > of surviving faults. There is no correlation to the total number of nodes > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > on the probability of a node failure. > > > > From the viewpoint of balancing load, it makes much sense. When setting > > replica number to the number of nodes, the cluster can balance search > > requests to all nodes which is optimal. > > > > Jörg > > > > On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > > > > > Hi all, > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > currently developing. > > > > > > This PoC models an application that needs durability but not > > > isolability, so I'm fine with the eventual consistency of reads against the > > > most recent writes. > > > > > > As durability is paramount (we can't affort to lose the data unless 100% > > > of the nodes die) I've been exploring the option of setting every shard to > > > have N replicas where N is the number of nodes in the cluster. > > > > > > From what I've read so far it is possible to dynamically set the number > > > or replicas which triggers a replication throttled replication process. > > > > > > I would like to have some help on the following steps (I'm running ES in > > > embedded mode in a Java application): > > > > > > 1 - How can I set the number or replicas using the native Java client ? > > > 2 - What happens if a node dies and the number of replicas is lowered to > > > the number of surviving ones? > > > 3 - Is it possible, from a participating node, to access the list of > > > nodes in the cluster so I can use their count to set the number of replicas > > > (step 1) ? > > > 4 - is it possible to hook a callback to the event of a node joining or > > > leaving the cluster ? > > > > > > I envisioning the following mechanism: > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > b)- Each time a node joins I adjust the number or replicas to match the > > > new node count. In this case, there would be 2 replicas > > > c) - An arbitrary number of nodes might be added and I'd execute step b) > > > accordingly > > > d) - At any time a node might leave the cluster and thus I need to lower > > > the number or replicas to the new node count (I assume that the cluster > > > would go ahead and proceed to compensate the lost replica by asking an > > > existing node to hold 2 replicas instead of one; is this stopped by > > > lowering the number or replicas?) > > > > > > The ultimate goal is to make sure no data is loss unless 100% of the > > > nodes die before a new one can acquire a full replica. > > > > > > Is this doable? Does this make sense at all ? > > > > > > For the time being, I'm not worried about lack of disk space or > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > Thank you very much for all your work and help. > > > > > > Gonçalo > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 10, 2014, 5:23pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/9 "2014-07-10T17:23:38Z")
For this, an async client implementation is needed that does not close connection after receiving a response, but waits continuously for a response stream (events), like I tried in the websocket transport plugin > **[GitHub - jprante/elasticsearch-transport-websocket: WebSockets for ElasticSearch](https://github.com/jprante/elasticsearch-transport-websocket)** > > WebSockets for ElasticSearch. Contribute to jprante/elasticsearch-transport-websocket development by creating an account on GitHub. It would be possible to attach a server-side service with ClusterStateListener to such a websocket channel for subscribing to cluster state events. Maybe I find time to implement this for demonstration. Jörg On Thu, Jul 10, 2014 at 7:03 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > That's what I discovered as well. I would love a client-side non-polling > mechanism. Even tried creating a local non-data/non-master node, but it > was not good enough. One day I will implement it as a plugin to ping an > external monitoring API, but that would be an overkill. > > Cheers, > > Ivan > > On Thu, Jul 10, 2014 at 9:55 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > On the client side, you can't use cluster state listener, it is for nodes > > that have access to a local copy of the master cluster state. Clients must > > execute an action to ask for cluster state, and with the current transport > > request/response cycle, they must poll for new events ... > > > > Jörg > > > > On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > Jörg, have you actually implemented your own ClusterStateListener? I > > > never had much success. Tried using that interface or > > > even PublishClusterStateAction.NewClusterStateListener, but either I could > > > not configure successfully the module (the former) or received no events > > > (the latter). Implemented on the client side, not as a plugin. > > > > > > Cheers, > > > > > > Ivan > > > > > > On Wed, Jul 9, 2014 at 4:21 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > > > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > > > > > 1. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf\_\_vo0gGh\_53gkXjKA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf\_\_vo0gGh\_53gkXjKA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 10, 2014, 6:06pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/10 "2014-07-10T18:06:51Z")
I have an idea of how to implement myself, I just don't have the bandwidth to support it. 🙂 I pretty much am the sole developer/architect/admin for our search project, so I am always hesitant to add another moving part. Will continue to poll for now. -- Ivan On Thu, Jul 10, 2014 at 10:23 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > For this, an async client implementation is needed that does not close > connection after receiving a response, but waits continuously for a > response stream (events), like I tried in the websocket transport plugin > [GitHub - jprante/elasticsearch-transport-websocket: WebSockets for ElasticSearch](https://github.com/jprante/elasticsearch-transport-websocket) > > It would be possible to attach a server-side service with > ClusterStateListener to such a websocket channel for subscribing to cluster > state events. Maybe I find time to implement this for demonstration. > > Jörg > > On Thu, Jul 10, 2014 at 7:03 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > That's what I discovered as well. I would love a client-side non-polling > > mechanism. Even tried creating a local non-data/non-master node, but it > > was not good enough. One day I will implement it as a plugin to ping an > > external monitoring API, but that would be an overkill. > > > > Cheers, > > > > Ivan > > > > On Thu, Jul 10, 2014 at 9:55 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > > > On the client side, you can't use cluster state listener, it is for > > > nodes that have access to a local copy of the master cluster state. Clients > > > must execute an action to ask for cluster state, and with the current > > > transport request/response cycle, they must poll for new events ... > > > > > > Jörg > > > > > > On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > > > Jörg, have you actually implemented your own ClusterStateListener? I > > > > never had much success. Tried using that interface or > > > > even PublishClusterStateAction.NewClusterStateListener, but either I could > > > > not configure successfully the module (the former) or received no events > > > > (the latter). Implemented on the client side, not as a plugin. > > > > > > > > Cheers, > > > > > > > > Ivan > > > > > > > > On Wed, Jul 9, 2014 at 4:21 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > > > > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > > > > > > > 1. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf\_\_vo0gGh\_53gkXjKA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf\_\_vo0gGh\_53gkXjKA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE%3DyhNyt8c8fGmqauUu7KfME3jqf__vo0gGh_53gkXjKA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDRdVYWfRnTrKtiZr%3DjvZU1CSzb06ijw-rFXntS%3Dg5Q1A%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHrkDe9%2BSuH3mLhTJYoPxvRBB8VY5nxPbsxsmc9S2Qr6g%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBrJS%3DDg0jP\_AOw21m-wP6qiTZfTAdTTDJ6%3DzP362\_0Fg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBrJS%3DDg0jP_AOw21m-wP6qiTZfTAdTTDJ6%3DzP362_0Fg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![goncalo\_luiz](https://avatars.discourse-cdn.com/v4/letter/g/e99b99/32.png) [@goncalo\_luiz](https://discuss.elastic.co/u/goncalo_luiz)\ **Post date:** [July 10, 2014, 7:39pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/11 "2014-07-10T19:39:16Z")
I get it know. I agree that setting the number of replicas is connected to the deployment reality in each case and it's derived variables and thus there is no one formula to fit all cases (it would't be a setting in that case). What I was trying to cover was the theoretical / extreme case where any node may fail at any time and what is the best way to go to minimize the chance of losing data. Also, in the case you want to scale down the installation (pottentially down to one node) without having to worry about selecting nodes that hold different replicated shards is an example that can beneffit from such configuration. I'm however not clear yet on what happens when a node goes down (triggering extra replication amongst the survivors) and then comes up again. Is the ongoing replication cancelled and the returning node brought up to date? Thanks for your valuable input. G. On 10 Jul 2014 18:07, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > All I say is that it depends on the probability of the event of three > nodes failing simultaneously, not on the total number of nodes having a > replica. You can even have 5 nodes and the probability of the event of 4 > nodes failing simultaneously, and so on. > > As an illustration, suppose you have a data center with two independent > electric circuits and the probability of failure corresponds with power > outage, then it is enough to distribute nodes equally over servers using > the two independent power lines in the racks. If one electric circuit (plus > UPS) fails, half of the nodes go down. With replica level 1, ES cluster > will keep all the data. There is no need to set replica level equal to node > number. > > Jörg > > On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > wrote: > > > Hi Joe, > > > > Thanks for your reply. > > On this thougth: > > > > " > > From my view your idea of better fault tolerance does not make much > > sense. The replica number is a statistical entity that is related to the > > probability of faults. The higher the replica, the higher the probability > > of surviving faults. There is no correlation to the total number of nodes > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > on the probability of a node failure." > > > > I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 of > > the nodes will have data of a given index (assuming 0 shards to ease the > > discussion), ritght? If those three nodes fail simultaneously the 4th will > > have no way of grabbing a copy and data will be lost forever. However if nr > > of replicas is 3, the 4th would be able to keep serving the requesrs and > > eventually handover a copy to a new node joining the cluster. > > How does this not help fault tolerance? I'm I missing something? > > > > Thanks, > > G. > > On 10 Jul 2014 00:21, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > wrote: > > > > > 1. You can set replica number at index creation time or by cluster > > > update settings > > > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > > > 2. You will get an index with lower replica number 🙂 > > > > > > 3. Yes. Quick code example: > > > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > From my view your idea of better fault tolerance does not make much > > > sense. The replica number is a statistical entity that is related to the > > > probability of faults. The higher the replica, the higher the probability > > > of surviving faults. There is no correlation to the total number of nodes > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > on the probability of a node failure. > > > > > > From the viewpoint of balancing load, it makes much sense. When setting > > > replica number to the number of nodes, the cluster can balance search > > > requests to all nodes which is optimal. > > > > > > Jörg > > > > > > On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > > > > > > > Hi all, > > > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > > currently developing. > > > > > > > > This PoC models an application that needs durability but not > > > > isolability, so I'm fine with the eventual consistency of reads against the > > > > most recent writes. > > > > > > > > As durability is paramount (we can't affort to lose the data unless > > > > 100% of the nodes die) I've been exploring the option of setting every > > > > shard to have N replicas where N is the number of nodes in the cluster. > > > > > > > > From what I've read so far it is possible to dynamically set the number > > > > or replicas which triggers a replication throttled replication process. > > > > > > > > I would like to have some help on the following steps (I'm running ES > > > > in embedded mode in a Java application): > > > > > > > > 1 - How can I set the number or replicas using the native Java client ? > > > > 2 - What happens if a node dies and the number of replicas is lowered > > > > to the number of surviving ones? > > > > 3 - Is it possible, from a participating node, to access the list of > > > > nodes in the cluster so I can use their count to set the number of replicas > > > > (step 1) ? > > > > 4 - is it possible to hook a callback to the event of a node joining or > > > > leaving the cluster ? > > > > > > > > I envisioning the following mechanism: > > > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > > b)- Each time a node joins I adjust the number or replicas to match the > > > > new node count. In this case, there would be 2 replicas > > > > c) - An arbitrary number of nodes might be added and I'd execute step > > > > b) accordingly > > > > d) - At any time a node might leave the cluster and thus I need to > > > > lower the number or replicas to the new node count (I assume that the > > > > cluster would go ahead and proceed to compensate the lost replica by asking > > > > an existing node to hold 2 replicas instead of one; is this stopped by > > > > lowering the number or replicas?) > > > > > > > > The ultimate goal is to make sure no data is loss unless 100% of the > > > > nodes die before a new one can acquire a full replica. > > > > > > > > Is this doable? Does this make sense at all ? > > > > > > > > For the time being, I'm not worried about lack of disk space or > > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > > > Thank you very much for all your work and help. > > > > > > > > Gonçalo > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 10, 2014, 7:57pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/12 "2014-07-10T19:57:47Z")
Sticking to your use case, you might want to use the auto\_expand\_replicas setting to "all" [1]: Never used it, but it sounds what you are looking for. By default, the ongoing recovery is not cancelled when the missing node rejoins the cluster. You can change the gateway settings [2] to control when recovery kicks in. [1] > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. [2] > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Cheers, Ivan On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > I get it know. > > I agree that setting the number of replicas is connected to the deployment > reality in each case and it's derived variables and thus there is no one > formula to fit all cases (it would't be a setting in that case). > > What I was trying to cover was the theoretical / extreme case where any > node may fail at any time and what is the best way to go to minimize the > chance of losing data. Also, in the case you want to scale down the > installation (pottentially down to one node) without having to worry about > selecting nodes that hold different replicated shards is an example that > can beneffit from such configuration. > > I'm however not clear yet on what happens when a node goes down > (triggering extra replication amongst the survivors) and then comes up > again. Is the ongoing replication cancelled and the returning node brought > up to date? > > Thanks for your valuable input. > > G. > On 10 Jul 2014 18:07, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > wrote: > > > All I say is that it depends on the probability of the event of three > > nodes failing simultaneously, not on the total number of nodes having a > > replica. You can even have 5 nodes and the probability of the event of 4 > > nodes failing simultaneously, and so on. > > > > As an illustration, suppose you have a data center with two independent > > electric circuits and the probability of failure corresponds with power > > outage, then it is enough to distribute nodes equally over servers using > > the two independent power lines in the racks. If one electric circuit (plus > > UPS) fails, half of the nodes go down. With replica level 1, ES cluster > > will keep all the data. There is no need to set replica level equal to node > > number. > > > > Jörg > > > > On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > > wrote: > > > > > Hi Joe, > > > > > > Thanks for your reply. > > > On this thougth: > > > > > > " > > > From my view your idea of better fault tolerance does not make much > > > sense. The replica number is a statistical entity that is related to the > > > probability of faults. The higher the replica, the higher the probability > > > of surviving faults. There is no correlation to the total number of nodes > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > on the probability of a node failure." > > > > > > I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 > > > of the nodes will have data of a given index (assuming 0 shards to ease the > > > discussion), ritght? If those three nodes fail simultaneously the 4th will > > > have no way of grabbing a copy and data will be lost forever. However if nr > > > of replicas is 3, the 4th would be able to keep serving the requesrs and > > > eventually handover a copy to a new node joining the cluster. > > > How does this not help fault tolerance? I'm I missing something? > > > > > > Thanks, > > > G. > > > On 10 Jul 2014 00:21, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > > wrote: > > > > > > > 1. You can set replica number at index creation time or by cluster > > > > update settings > > > > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > > > > > 2. You will get an index with lower replica number 🙂 > > > > > > > > 3. Yes. Quick code example: > > > > > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > From my view your idea of better fault tolerance does not make much > > > > sense. The replica number is a statistical entity that is related to the > > > > probability of faults. The higher the replica, the higher the probability > > > > of surviving faults. There is no correlation to the total number of nodes > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > on the probability of a node failure. > > > > > > > > From the viewpoint of balancing load, it makes much sense. When setting > > > > replica number to the number of nodes, the cluster can balance search > > > > requests to all nodes which is optimal. > > > > > > > > Jörg > > > > > > > > On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > > > > > > > > > Hi all, > > > > > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > > > currently developing. > > > > > > > > > > This PoC models an application that needs durability but not > > > > > isolability, so I'm fine with the eventual consistency of reads against the > > > > > most recent writes. > > > > > > > > > > As durability is paramount (we can't affort to lose the data unless > > > > > 100% of the nodes die) I've been exploring the option of setting every > > > > > shard to have N replicas where N is the number of nodes in the cluster. > > > > > > > > > > From what I've read so far it is possible to dynamically set the > > > > > number or replicas which triggers a replication throttled replication > > > > > process. > > > > > > > > > > I would like to have some help on the following steps (I'm running ES > > > > > in embedded mode in a Java application): > > > > > > > > > > 1 - How can I set the number or replicas using the native Java client ? > > > > > 2 - What happens if a node dies and the number of replicas is lowered > > > > > to the number of surviving ones? > > > > > 3 - Is it possible, from a participating node, to access the list of > > > > > nodes in the cluster so I can use their count to set the number of replicas > > > > > (step 1) ? > > > > > 4 - is it possible to hook a callback to the event of a node joining > > > > > or leaving the cluster ? > > > > > > > > > > I envisioning the following mechanism: > > > > > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > > > b)- Each time a node joins I adjust the number or replicas to match > > > > > the new node count. In this case, there would be 2 replicas > > > > > c) - An arbitrary number of nodes might be added and I'd execute step > > > > > b) accordingly > > > > > d) - At any time a node might leave the cluster and thus I need to > > > > > lower the number or replicas to the new node count (I assume that the > > > > > cluster would go ahead and proceed to compensate the lost replica by asking > > > > > an existing node to hold 2 replicas instead of one; is this stopped by > > > > > lowering the number or replicas?) > > > > > > > > > > The ultimate goal is to make sure no data is loss unless 100% of the > > > > > nodes die before a new one can acquire a full replica. > > > > > > > > > > Is this doable? Does this make sense at all ? > > > > > > > > > > For the time being, I'm not worried about lack of disk space or > > > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > > > > > Thank you very much for all your work and help. > > > > > > > > > > Gonçalo > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 10, 2014, 8:48pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/13 "2014-07-10T20:48:07Z")
Indeed, auto\_expand\_replicas "all" triggers an update cluster settings action each time a node is added. Still blown by the many settings Elasticsearch provides. Feeling small. Homework: collecting a gist textfile of all ES 1.2 settings. Jörg On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > Sticking to your use case, you might want to use the auto\_expand\_replicas > setting to "all" [1]: Never used it, but it sounds what you are looking for. > > By default, the ongoing recovery is not cancelled when the missing node > rejoins the cluster. You can change the gateway settings [2] to control > when recovery kicks in. > > [1] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html) > [2] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html) > > Cheers, > > Ivan > > On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > wrote: > > > I get it know. > > > > I agree that setting the number of replicas is connected to the > > deployment reality in each case and it's derived variables and thus there > > is no one formula to fit all cases (it would't be a setting in that case). > > > > What I was trying to cover was the theoretical / extreme case where any > > node may fail at any time and what is the best way to go to minimize the > > chance of losing data. Also, in the case you want to scale down the > > installation (pottentially down to one node) without having to worry about > > selecting nodes that hold different replicated shards is an example that > > can beneffit from such configuration. > > > > I'm however not clear yet on what happens when a node goes down > > (triggering extra replication amongst the survivors) and then comes up > > again. Is the ongoing replication cancelled and the returning node brought > > up to date? > > > > Thanks for your valuable input. > > > > G. > > On 10 Jul 2014 18:07, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > wrote: > > > > > All I say is that it depends on the probability of the event of three > > > nodes failing simultaneously, not on the total number of nodes having a > > > replica. You can even have 5 nodes and the probability of the event of 4 > > > nodes failing simultaneously, and so on. > > > > > > As an illustration, suppose you have a data center with two independent > > > electric circuits and the probability of failure corresponds with power > > > outage, then it is enough to distribute nodes equally over servers using > > > the two independent power lines in the racks. If one electric circuit (plus > > > UPS) fails, half of the nodes go down. With replica level 1, ES cluster > > > will keep all the data. There is no need to set replica level equal to node > > > number. > > > > > > Jörg > > > > > > On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > > > wrote: > > > > > > > Hi Joe, > > > > > > > > Thanks for your reply. > > > > On this thougth: > > > > > > > > " > > > > From my view your idea of better fault tolerance does not make much > > > > sense. The replica number is a statistical entity that is related to the > > > > probability of faults. The higher the replica, the higher the probability > > > > of surviving faults. There is no correlation to the total number of nodes > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > on the probability of a node failure." > > > > > > > > I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 > > > > of the nodes will have data of a given index (assuming 0 shards to ease the > > > > discussion), ritght? If those three nodes fail simultaneously the 4th will > > > > have no way of grabbing a copy and data will be lost forever. However if nr > > > > of replicas is 3, the 4th would be able to keep serving the requesrs and > > > > eventually handover a copy to a new node joining the cluster. > > > > How does this not help fault tolerance? I'm I missing something? > > > > > > > > Thanks, > > > > G. > > > > On 10 Jul 2014 00:21, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > > > wrote: > > > > > > > > > 1. You can set replica number at index creation time or by cluster > > > > > update settings > > > > > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > > > > > > > 2. You will get an index with lower replica number 🙂 > > > > > > > > > > 3. Yes. Quick code example: > > > > > > > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > > > From my view your idea of better fault tolerance does not make much > > > > > sense. The replica number is a statistical entity that is related to the > > > > > probability of faults. The higher the replica, the higher the probability > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > on the probability of a node failure. > > > > > > > > > > From the viewpoint of balancing load, it makes much sense. When > > > > > setting replica number to the number of nodes, the cluster can balance > > > > > search requests to all nodes which is optimal. > > > > > > > > > > Jörg > > > > > > > > > > On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > > > > currently developing. > > > > > > > > > > > > This PoC models an application that needs durability but not > > > > > > isolability, so I'm fine with the eventual consistency of reads against the > > > > > > most recent writes. > > > > > > > > > > > > As durability is paramount (we can't affort to lose the data unless > > > > > > 100% of the nodes die) I've been exploring the option of setting every > > > > > > shard to have N replicas where N is the number of nodes in the cluster. > > > > > > > > > > > > From what I've read so far it is possible to dynamically set the > > > > > > number or replicas which triggers a replication throttled replication > > > > > > process. > > > > > > > > > > > > I would like to have some help on the following steps (I'm running ES > > > > > > in embedded mode in a Java application): > > > > > > > > > > > > 1 - How can I set the number or replicas using the native Java client > > > > > > ? > > > > > > 2 - What happens if a node dies and the number of replicas is lowered > > > > > > to the number of surviving ones? > > > > > > 3 - Is it possible, from a participating node, to access the list of > > > > > > nodes in the cluster so I can use their count to set the number of replicas > > > > > > (step 1) ? > > > > > > 4 - is it possible to hook a callback to the event of a node joining > > > > > > or leaving the cluster ? > > > > > > > > > > > > I envisioning the following mechanism: > > > > > > > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > > > > b)- Each time a node joins I adjust the number or replicas to match > > > > > > the new node count. In this case, there would be 2 replicas > > > > > > c) - An arbitrary number of nodes might be added and I'd execute step > > > > > > b) accordingly > > > > > > d) - At any time a node might leave the cluster and thus I need to > > > > > > lower the number or replicas to the new node count (I assume that the > > > > > > cluster would go ahead and proceed to compensate the lost replica by asking > > > > > > an existing node to hold 2 replicas instead of one; is this stopped by > > > > > > lowering the number or replicas?) > > > > > > > > > > > > The ultimate goal is to make sure no data is loss unless 100% of the > > > > > > nodes die before a new one can acquire a full replica. > > > > > > > > > > > > Is this doable? Does this make sense at all ? > > > > > > > > > > > > For the time being, I'm not worried about lack of disk space or > > > > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > > > > > > > Thank you very much for all your work and help. > > > > > > > > > > > > Gonçalo > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 10, 2014, 9:11pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/14 "2014-07-10T21:11:31Z")
It's only been around for 3.5 years: [A setting to auto expand the number of replicas of an index (based on data nodes) · Issue #623 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/623) 🙂 I should clarify part of my previous statement. _"By default, the ongoing recovery is not cancelled when the missing node rejoins the cluster. You can change the gateway settings [2] to control when recovery kicks in."_ What I meant to say is that an ongoing recovery is never cancelled once it has commenced, no matter what settings. By default, recovery happens immediately, but can be changed with the gateway settings. -- Ivan On Thu, Jul 10, 2014 at 1:48 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > Indeed, auto\_expand\_replicas "all" triggers an update cluster settings > action each time a node is added. > > Still blown by the many settings Elasticsearch provides. Feeling small. > Homework: collecting a gist textfile of all ES 1.2 settings. > > Jörg > > On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > Sticking to your use case, you might want to use the auto\_expand\_replicas > > setting to "all" [1]: Never used it, but it sounds what you are looking for. > > > > By default, the ongoing recovery is not cancelled when the missing node > > rejoins the cluster. You can change the gateway settings [2] to control > > when recovery kicks in. > > > > [1] > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html) > > [2] > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html) > > > > Cheers, > > > > Ivan > > > > On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > > wrote: > > > > > I get it know. > > > > > > I agree that setting the number of replicas is connected to the > > > deployment reality in each case and it's derived variables and thus there > > > is no one formula to fit all cases (it would't be a setting in that case). > > > > > > What I was trying to cover was the theoretical / extreme case where any > > > node may fail at any time and what is the best way to go to minimize the > > > chance of losing data. Also, in the case you want to scale down the > > > installation (pottentially down to one node) without having to worry about > > > selecting nodes that hold different replicated shards is an example that > > > can beneffit from such configuration. > > > > > > I'm however not clear yet on what happens when a node goes down > > > (triggering extra replication amongst the survivors) and then comes up > > > again. Is the ongoing replication cancelled and the returning node brought > > > up to date? > > > > > > Thanks for your valuable input. > > > > > > G. > > > On 10 Jul 2014 18:07, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > > wrote: > > > > > > > All I say is that it depends on the probability of the event of three > > > > nodes failing simultaneously, not on the total number of nodes having a > > > > replica. You can even have 5 nodes and the probability of the event of 4 > > > > nodes failing simultaneously, and so on. > > > > > > > > As an illustration, suppose you have a data center with two independent > > > > electric circuits and the probability of failure corresponds with power > > > > outage, then it is enough to distribute nodes equally over servers using > > > > the two independent power lines in the racks. If one electric circuit (plus > > > > UPS) fails, half of the nodes go down. With replica level 1, ES cluster > > > > will keep all the data. There is no need to set replica level equal to node > > > > number. > > > > > > > > Jörg > > > > > > > > On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > > > > wrote: > > > > > > > > > Hi Joe, > > > > > > > > > > Thanks for your reply. > > > > > On this thougth: > > > > > > > > > > " > > > > > From my view your idea of better fault tolerance does not make much > > > > > sense. The replica number is a statistical entity that is related to the > > > > > probability of faults. The higher the replica, the higher the probability > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > on the probability of a node failure." > > > > > > > > > > I'm not getting it. If we have 4 nodes with 2 replicas it means that 3 > > > > > of the nodes will have data of a given index (assuming 0 shards to ease the > > > > > discussion), ritght? If those three nodes fail simultaneously the 4th will > > > > > have no way of grabbing a copy and data will be lost forever. However if nr > > > > > of replicas is 3, the 4th would be able to keep serving the requesrs and > > > > > eventually handover a copy to a new node joining the cluster. > > > > > How does this not help fault tolerance? I'm I missing something? > > > > > > > > > > Thanks, > > > > > G. > > > > > On 10 Jul 2014 00:21, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > > > > wrote: > > > > > > > > > > > 1. You can set replica number at index creation time or by cluster > > > > > > update settings > > > > > > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > > > > > > > > > 2. You will get an index with lower replica number 🙂 > > > > > > > > > > > > 3. Yes. Quick code example: > > > > > > > > > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > > > > > From my view your idea of better fault tolerance does not make much > > > > > > sense. The replica number is a statistical entity that is related to the > > > > > > probability of faults. The higher the replica, the higher the probability > > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > > on the probability of a node failure. > > > > > > > > > > > > From the viewpoint of balancing load, it makes much sense. When > > > > > > setting replica number to the number of nodes, the cluster can balance > > > > > > search requests to all nodes which is optimal. > > > > > > > > > > > > Jörg > > > > > > > > > > > > On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > > > > > currently developing. > > > > > > > > > > > > > > This PoC models an application that needs durability but not > > > > > > > isolability, so I'm fine with the eventual consistency of reads against the > > > > > > > most recent writes. > > > > > > > > > > > > > > As durability is paramount (we can't affort to lose the data unless > > > > > > > 100% of the nodes die) I've been exploring the option of setting every > > > > > > > shard to have N replicas where N is the number of nodes in the cluster. > > > > > > > > > > > > > > From what I've read so far it is possible to dynamically set the > > > > > > > number or replicas which triggers a replication throttled replication > > > > > > > process. > > > > > > > > > > > > > > I would like to have some help on the following steps (I'm running > > > > > > > ES in embedded mode in a Java application): > > > > > > > > > > > > > > 1 - How can I set the number or replicas using the native Java > > > > > > > client ? > > > > > > > 2 - What happens if a node dies and the number of replicas is > > > > > > > lowered to the number of surviving ones? > > > > > > > 3 - Is it possible, from a participating node, to access the list of > > > > > > > nodes in the cluster so I can use their count to set the number of replicas > > > > > > > (step 1) ? > > > > > > > 4 - is it possible to hook a callback to the event of a node joining > > > > > > > or leaving the cluster ? > > > > > > > > > > > > > > I envisioning the following mechanism: > > > > > > > > > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > > > > > b)- Each time a node joins I adjust the number or replicas to match > > > > > > > the new node count. In this case, there would be 2 replicas > > > > > > > c) - An arbitrary number of nodes might be added and I'd execute > > > > > > > step b) accordingly > > > > > > > d) - At any time a node might leave the cluster and thus I need to > > > > > > > lower the number or replicas to the new node count (I assume that the > > > > > > > cluster would go ahead and proceed to compensate the lost replica by asking > > > > > > > an existing node to hold 2 replicas instead of one; is this stopped by > > > > > > > lowering the number or replicas?) > > > > > > > > > > > > > > The ultimate goal is to make sure no data is loss unless 100% of the > > > > > > > nodes die before a new one can acquire a full replica. > > > > > > > > > > > > > > Is this doable? Does this make sense at all ? > > > > > > > > > > > > > > For the time being, I'm not worried about lack of disk space or > > > > > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > > > > > > > > > Thank you very much for all your work and help. > > > > > > > > > > > > > > Gonçalo > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit > > > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA\_LtOdS-6Ht\_DR57P%2BXkLWp5%3DV5Dz%2BVh2\_cMkgy6kDSw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![goncalo\_luiz](https://avatars.discourse-cdn.com/v4/letter/g/e99b99/32.png) [@goncalo\_luiz](https://discuss.elastic.co/u/goncalo_luiz)\ **Post date:** [July 11, 2014, 6:00am UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/15 "2014-07-11T06:00:14Z")
Hi Ivan, Does this mean that if a note comes back and a replication is underway we'll end up with a node holding 2 replicas and 1 node holding node ? Scenario: Node A - Replica 2 Node B - Replica 3 Node C - Replica 1 If node A dies and Node B get's Replica 2, as soon as node A (or a replacement) is brought up, is the final configuration likely to be Node A (or replcament) - No replicas Node B .- Replica 3 and 2 Node C - Replica 1 or is there a re-balance that takes place ? Thanks, Gonçalo Gonçalo Luiz On 10 July 2014 22:11, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > It's only been around for 3.5 years: > [A setting to auto expand the number of replicas of an index (based on data nodes) · Issue #623 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/623) 🙂 > > I should clarify part of my previous statement. > > _"By default, the ongoing recovery is not cancelled when the missing node > rejoins the cluster. You can change the gateway settings [2] to control > when recovery kicks in."_ > > What I meant to say is that an ongoing recovery is never cancelled once it > has commenced, no matter what settings. By default, recovery happens > immediately, but can be changed with the gateway settings. > > -- > Ivan > > On Thu, Jul 10, 2014 at 1:48 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > Indeed, auto\_expand\_replicas "all" triggers an update cluster settings > > action each time a node is added. > > > > Still blown by the many settings Elasticsearch provides. Feeling small. > > Homework: collecting a gist textfile of all ES 1.2 settings. > > > > Jörg > > > > On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > Sticking to your use case, you might want to use > > > the auto\_expand\_replicas setting to "all" [1]: Never used it, but it sounds > > > what you are looking for. > > > > > > By default, the ongoing recovery is not cancelled when the missing node > > > rejoins the cluster. You can change the gateway settings [2] to control > > > when recovery kicks in. > > > > > > [1] > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html) > > > [2] > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html) > > > > > > Cheers, > > > > > > Ivan > > > > > > On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > > > wrote: > > > > > > > I get it know. > > > > > > > > I agree that setting the number of replicas is connected to the > > > > deployment reality in each case and it's derived variables and thus there > > > > is no one formula to fit all cases (it would't be a setting in that case). > > > > > > > > What I was trying to cover was the theoretical / extreme case where any > > > > node may fail at any time and what is the best way to go to minimize the > > > > chance of losing data. Also, in the case you want to scale down the > > > > installation (pottentially down to one node) without having to worry about > > > > selecting nodes that hold different replicated shards is an example that > > > > can beneffit from such configuration. > > > > > > > > I'm however not clear yet on what happens when a node goes down > > > > (triggering extra replication amongst the survivors) and then comes up > > > > again. Is the ongoing replication cancelled and the returning node brought > > > > up to date? > > > > > > > > Thanks for your valuable input. > > > > > > > > G. > > > > On 10 Jul 2014 18:07, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > > > wrote: > > > > > > > > > All I say is that it depends on the probability of the event of three > > > > > nodes failing simultaneously, not on the total number of nodes having a > > > > > replica. You can even have 5 nodes and the probability of the event of 4 > > > > > nodes failing simultaneously, and so on. > > > > > > > > > > As an illustration, suppose you have a data center with two > > > > > independent electric circuits and the probability of failure corresponds > > > > > with power outage, then it is enough to distribute nodes equally over > > > > > servers using the two independent power lines in the racks. If one electric > > > > > circuit (plus UPS) fails, half of the nodes go down. With replica level 1, > > > > > ES cluster will keep all the data. There is no need to set replica level > > > > > equal to node number. > > > > > > > > > > Jörg > > > > > > > > > > On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) > > > > > wrote: > > > > > > > > > > > Hi Joe, > > > > > > > > > > > > Thanks for your reply. > > > > > > On this thougth: > > > > > > > > > > > > " > > > > > > From my view your idea of better fault tolerance does not make much > > > > > > sense. The replica number is a statistical entity that is related to the > > > > > > probability of faults. The higher the replica, the higher the probability > > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > > on the probability of a node failure." > > > > > > > > > > > > I'm not getting it. If we have 4 nodes with 2 replicas it means that > > > > > > 3 of the nodes will have data of a given index (assuming 0 shards to ease > > > > > > the discussion), ritght? If those three nodes fail simultaneously the 4th > > > > > > will have no way of grabbing a copy and data will be lost forever. However > > > > > > if nr of replicas is 3, the 4th would be able to keep serving the requesrs > > > > > > and eventually handover a copy to a new node joining the cluster. > > > > > > How does this not help fault tolerance? I'm I missing something? > > > > > > > > > > > > Thanks, > > > > > > G. > > > > > > On 10 Jul 2014 00:21, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > > > > > wrote: > > > > > > > > > > > > > 1. You can set replica number at index creation time or by cluster > > > > > > > update settings > > > > > > > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > > > > > > > > > > > 2. You will get an index with lower replica number 🙂 > > > > > > > > > > > > > > 3. Yes. Quick code example: > > > > > > > > > > > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > > > > > > > From my view your idea of better fault tolerance does not make much > > > > > > > sense. The replica number is a statistical entity that is related to the > > > > > > > probability of faults. The higher the replica, the higher the probability > > > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > > > on the probability of a node failure. > > > > > > > > > > > > > > From the viewpoint of balancing load, it makes much sense. When > > > > > > > setting replica number to the number of nodes, the cluster can balance > > > > > > > search requests to all nodes which is optimal. > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > On Wed, Jul 9, 2014 at 11:57 PM, [goncalo.luiz@gmail.com](mailto:goncalo.luiz@gmail.com) wrote: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > > > > > > currently developing. > > > > > > > > > > > > > > > > This PoC models an application that needs durability but not > > > > > > > > isolability, so I'm fine with the eventual consistency of reads against the > > > > > > > > most recent writes. > > > > > > > > > > > > > > > > As durability is paramount (we can't affort to lose the data unless > > > > > > > > 100% of the nodes die) I've been exploring the option of setting every > > > > > > > > shard to have N replicas where N is the number of nodes in the cluster. > > > > > > > > > > > > > > > > From what I've read so far it is possible to dynamically set the > > > > > > > > number or replicas which triggers a replication throttled replication > > > > > > > > process. > > > > > > > > > > > > > > > > I would like to have some help on the following steps (I'm running > > > > > > > > ES in embedded mode in a Java application): > > > > > > > > > > > > > > > > 1 - How can I set the number or replicas using the native Java > > > > > > > > client ? > > > > > > > > 2 - What happens if a node dies and the number of replicas is > > > > > > > > lowered to the number of surviving ones? > > > > > > > > 3 - Is it possible, from a participating node, to access the list > > > > > > > > of nodes in the cluster so I can use their count to set the number of > > > > > > > > replicas (step 1) ? > > > > > > > > 4 - is it possible to hook a callback to the event of a node > > > > > > > > joining or leaving the cluster ? > > > > > > > > > > > > > > > > I envisioning the following mechanism: > > > > > > > > > > > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > > > > > > b)- Each time a node joins I adjust the number or replicas to match > > > > > > > > the new node count. In this case, there would be 2 replicas > > > > > > > > c) - An arbitrary number of nodes might be added and I'd execute > > > > > > > > step b) accordingly > > > > > > > > d) - At any time a node might leave the cluster and thus I need to > > > > > > > > lower the number or replicas to the new node count (I assume that the > > > > > > > > cluster would go ahead and proceed to compensate the lost replica by asking > > > > > > > > an existing node to hold 2 replicas instead of one; is this stopped by > > > > > > > > lowering the number or replicas?) > > > > > > > > > > > > > > > > The ultimate goal is to make sure no data is loss unless 100% of > > > > > > > > the nodes die before a new one can acquire a full replica. > > > > > > > > > > > > > > > > Is this doable? Does this make sense at all ? > > > > > > > > > > > > > > > > For the time being, I'm not worried about lack of disk space or > > > > > > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > > > > > > > > > > > Thank you very much for all your work and help. > > > > > > > > > > > > > > > > Gonçalo > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > To unsubscribe from this topic, visit > > > > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > > > > . > > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA\_LtOdS-6Ht\_DR57P%2BXkLWp5%3DV5Dz%2BVh2\_cMkgy6kDSw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA\_LtOdS-6Ht\_DR57P%2BXkLWp5%3DV5Dz%2BVh2\_cMkgy6kDSw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMrm09rNQGaaMLE0iOD3NbFWNo7FtmzLh-JswwcKr3Ggp0ztfg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09rNQGaaMLE0iOD3NbFWNo7FtmzLh-JswwcKr3Ggp0ztfg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Glen\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glen_smith/32/111656_2.png) [@Glen\_Smith](https://discuss.elastic.co/u/Glen_Smith)\ **Post date:** [July 11, 2014, 11:11am UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/16 "2014-07-11T11:11:20Z")
Hi Goncalo, I think it's important that you understand: multiple copies of a shard will never be located on the same node. Not two replicas, and not the primary and one replica. To witness this, run a server on your local machine, and create an index with the defaults - 5 shards, one replica. You will see that your cluster is "yellow", and has 5 unallocated shards. How that helps create a better mental picture of shard allocation. On Friday, July 11, 2014 2:00:47 AM UTC-4, Gonçalo Luiz wrote: > Hi Ivan, > > Does this mean that if a note comes back and a replication is underway > we'll end up with a node holding 2 replicas and 1 node holding node ? > > Scenario: > > Node A - Replica 2 > Node B - Replica 3 > Node C - Replica 1 > > If node A dies and Node B get's Replica 2, as soon as node A (or a > replacement) is brought up, is the final configuration likely to be > > Node A (or replcament) - No replicas > Node B .- Replica 3 and 2 > Node C - Replica 1 > > or is there a re-balance that takes place ? > > Thanks, > Gonçalo > > Gonçalo Luiz > > On 10 July 2014 22:11, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com) \\> wrote: > > > It's only been around for 3.5 years: > > [A setting to auto expand the number of replicas of an index (based on data nodes) · Issue #623 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/623) 🙂 > > > > I should clarify part of my previous statement. > > > > _"By default, the ongoing recovery is not cancelled when the missing node > > rejoins the cluster. You can change the gateway settings [2] to control > > when recovery kicks in."_ > > > > What I meant to say is that an ongoing recovery is never cancelled once > > it has commenced, no matter what settings. By default, recovery happens > > immediately, but can be changed with the gateway settings. > > > > -- > > Ivan > > > > On Thu, Jul 10, 2014 at 1:48 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) \ \< > > [joerg...@gmail.com](mailto:joerg...@gmail.com) \\> wrote: > > > > > Indeed, auto\_expand\_replicas "all" triggers an update cluster settings > > > action each time a node is added. > > > > > > Still blown by the many settings Elasticsearch provides. Feeling small. > > > Homework: collecting a gist textfile of all ES 1.2 settings. > > > > > > Jörg > > > > > > On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com) > > > \\> wrote: > > > > > > > Sticking to your use case, you might want to use > > > > the auto\_expand\_replicas setting to "all" [1]: Never used it, but it sounds > > > > what you are looking for. > > > > > > > > By default, the ongoing recovery is not cancelled when the missing node > > > > rejoins the cluster. You can change the gateway settings [2] to control > > > > when recovery kicks in. > > > > > > > > [1] > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html) > > > > [2] > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html) > > > > > > > > Cheers, > > > > > > > > Ivan > > > > > > > > On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz \<[goncal...@gmail.com](mailto:goncal...@gmail.com) > > > > \\> wrote: > > > > > > > > > I get it know. > > > > > > > > > > I agree that setting the number of replicas is connected to the > > > > > deployment reality in each case and it's derived variables and thus there > > > > > is no one formula to fit all cases (it would't be a setting in that case). > > > > > > > > > > What I was trying to cover was the theoretical / extreme case where > > > > > any node may fail at any time and what is the best way to go to minimize > > > > > the chance of losing data. Also, in the case you want to scale down the > > > > > installation (pottentially down to one node) without having to worry about > > > > > selecting nodes that hold different replicated shards is an example that > > > > > can beneffit from such configuration. > > > > > > > > > > I'm however not clear yet on what happens when a node goes down > > > > > (triggering extra replication amongst the survivors) and then comes up > > > > > again. Is the ongoing replication cancelled and the returning node brought > > > > > up to date? > > > > > > > > > > Thanks for your valuable input. > > > > > > > > > > G. > > > > > On 10 Jul 2014 18:07, "[joerg...@gmail.com](mailto:joerg...@gmail.com) \" \< > > > > > [joerg...@gmail.com](mailto:joerg...@gmail.com) \\> wrote: > > > > > > > > > > > All I say is that it depends on the probability of the event of three > > > > > > nodes failing simultaneously, not on the total number of nodes having a > > > > > > replica. You can even have 5 nodes and the probability of the event of 4 > > > > > > nodes failing simultaneously, and so on. > > > > > > > > > > > > As an illustration, suppose you have a data center with two > > > > > > independent electric circuits and the probability of failure corresponds > > > > > > with power outage, then it is enough to distribute nodes equally over > > > > > > servers using the two independent power lines in the racks. If one electric > > > > > > circuit (plus UPS) fails, half of the nodes go down. With replica level 1, > > > > > > ES cluster will keep all the data. There is no need to set replica level > > > > > > equal to node number. > > > > > > > > > > > > Jörg > > > > > > > > > > > > On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz \<[goncal...@gmail.com](mailto:goncal...@gmail.com) > > > > > > \\> wrote: > > > > > > > > > > > > > Hi Joe, > > > > > > > > > > > > > > Thanks for your reply. > > > > > > > On this thougth: > > > > > > > > > > > > > > " > > > > > > > From my view your idea of better fault tolerance does not make much > > > > > > > sense. The replica number is a statistical entity that is related to the > > > > > > > probability of faults. The higher the replica, the higher the probability > > > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > > > on the probability of a node failure." > > > > > > > > > > > > > > I'm not getting it. If we have 4 nodes with 2 replicas it means that > > > > > > > 3 of the nodes will have data of a given index (assuming 0 shards to ease > > > > > > > the discussion), ritght? If those three nodes fail simultaneously the 4th > > > > > > > will have no way of grabbing a copy and data will be lost forever. However > > > > > > > if nr of replicas is 3, the 4th would be able to keep serving the requesrs > > > > > > > and eventually handover a copy to a new node joining the cluster. > > > > > > > How does this not help fault tolerance? I'm I missing something? > > > > > > > > > > > > > > Thanks, > > > > > > > G. > > > > > > > On 10 Jul 2014 00:21, "[joerg...@gmail.com](mailto:joerg...@gmail.com) \" \< > > > > > > > [joerg...@gmail.com](mailto:joerg...@gmail.com) \\> wrote: > > > > > > > > > > > > > > > 1. You can set replica number at index creation time or by cluster > > > > > > > > update settings > > > > > > > > action org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > > > > > > > > > > > > > 2. You will get an index with lower replica number 🙂 > > > > > > > > > > > > > > > > 3. Yes. Quick code example: > > > > > > > > > > > > > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > > > > > > > > > From my view your idea of better fault tolerance does not make much > > > > > > > > sense. The replica number is a statistical entity that is related to the > > > > > > > > probability of faults. The higher the replica, the higher the probability > > > > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > > > > on the probability of a node failure. > > > > > > > > > > > > > > > > From the viewpoint of balancing load, it makes much sense. When > > > > > > > > setting replica number to the number of nodes, the cluster can balance > > > > > > > > search requests to all nodes which is optimal. > > > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > > > On Wed, Jul 9, 2014 at 11:57 PM, \<[goncal...@gmail.com](mailto:goncal...@gmail.com) \ > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > > > > > > > currently developing. > > > > > > > > > > > > > > > > > > This PoC models an application that needs durability but not > > > > > > > > > isolability, so I'm fine with the eventual consistency of reads against the > > > > > > > > > most recent writes. > > > > > > > > > > > > > > > > > > As durability is paramount (we can't affort to lose the data > > > > > > > > > unless 100% of the nodes die) I've been exploring the option of setting > > > > > > > > > every shard to have N replicas where N is the number of nodes in the > > > > > > > > > cluster. > > > > > > > > > > > > > > > > > > From what I've read so far it is possible to dynamically set the > > > > > > > > > number or replicas which triggers a replication throttled replication > > > > > > > > > process. > > > > > > > > > > > > > > > > > > I would like to have some help on the following steps (I'm running > > > > > > > > > ES in embedded mode in a Java application): > > > > > > > > > > > > > > > > > > 1 - How can I set the number or replicas using the native Java > > > > > > > > > client ? > > > > > > > > > 2 - What happens if a node dies and the number of replicas is > > > > > > > > > lowered to the number of surviving ones? > > > > > > > > > 3 - Is it possible, from a participating node, to access the list > > > > > > > > > of nodes in the cluster so I can use their count to set the number of > > > > > > > > > replicas (step 1) ? > > > > > > > > > 4 - is it possible to hook a callback to the event of a node > > > > > > > > > joining or leaving the cluster ? > > > > > > > > > > > > > > > > > > I envisioning the following mechanism: > > > > > > > > > > > > > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > > > > > > > b)- Each time a node joins I adjust the number or replicas to > > > > > > > > > match the new node count. In this case, there would be 2 replicas > > > > > > > > > c) - An arbitrary number of nodes might be added and I'd execute > > > > > > > > > step b) accordingly > > > > > > > > > d) - At any time a node might leave the cluster and thus I need to > > > > > > > > > lower the number or replicas to the new node count (I assume that the > > > > > > > > > cluster would go ahead and proceed to compensate the lost replica by asking > > > > > > > > > an existing node to hold 2 replicas instead of one; is this stopped by > > > > > > > > > lowering the number or replicas?) > > > > > > > > > > > > > > > > > > The ultimate goal is to make sure no data is loss unless 100% of > > > > > > > > > the nodes die before a new one can acquire a full replica. > > > > > > > > > > > > > > > > > > Is this doable? Does this make sense at all ? > > > > > > > > > > > > > > > > > > For the time being, I'm not worried about lack of disk space or > > > > > > > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > > > > > > > > > > > > > Thank you very much for all your work and help. > > > > > > > > > > > > > > > > > > Gonçalo > > > > > > > > > > > > > > > > > > -- > > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > > Groups "elasticsearch" group. > > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > > > > To view this discussion on the web visit > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com) > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > > > . > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this topic, visit > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > > > > > . > > > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com) > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > > > . > > > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit > > > > > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe](https://groups.google.com/d/topic/elasticsearch/hPvVz20v6YY/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA\_LtOdS-6Ht\_DR57P%2BXkLWp5%3DV5Dz%2BVh2\_cMkgy6kDSw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA\_LtOdS-6Ht\_DR57P%2BXkLWp5%3DV5Dz%2BVh2\_cMkgy6kDSw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7a3b23e6-7e63-4ca3-9a22-4519353a5d2e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7a3b23e6-7e63-4ca3-9a22-4519353a5d2e%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![goncalo\_luiz](https://avatars.discourse-cdn.com/v4/letter/g/e99b99/32.png) [@goncalo\_luiz](https://discuss.elastic.co/u/goncalo_luiz)\ **Post date:** [July 11, 2014, 6:36pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/17 "2014-07-11T18:36:35Z")
Thanks for the clear and simple explanation. However, will the cluster (with auto expand replicas) ever go green if it has been grown from 2 to 3 (triggering replicas to grow to to) and then downsized to two nodes again? In other words, do the auto grow replicas setting work both ways or just upwards? Thanks again. G. On 11 Jul 2014 12:11, "Glen Smith" [glen@smithsrock.com](mailto:glen@smithsrock.com) wrote: > Hi Goncalo, > > I think it's important that you understand: multiple copies of a shard > will never be located on the same node. > Not two replicas, and not the primary and one replica. > To witness this, run a server on your local machine, and create an index > with the defaults - 5 shards, one replica. > You will see that your cluster is "yellow", and has 5 unallocated shards. > > How that helps create a better mental picture of shard allocation. > > On Friday, July 11, 2014 2:00:47 AM UTC-4, Gonçalo Luiz wrote: > > > Hi Ivan, > > > > Does this mean that if a note comes back and a replication is underway > > we'll end up with a node holding 2 replicas and 1 node holding node ? > > > > Scenario: > > > > Node A - Replica 2 > > Node B - Replica 3 > > Node C - Replica 1 > > > > If node A dies and Node B get's Replica 2, as soon as node A (or a > > replacement) is brought up, is the final configuration likely to be > > > > Node A (or replcament) - No replicas > > Node B .- Replica 3 and 2 > > Node C - Replica 1 > > > > or is there a re-balance that takes place ? > > > > Thanks, > > Gonçalo > > > > Gonçalo Luiz > > > > On 10 July 2014 22:11, Ivan Brusic [iv...@brusic.com](mailto:iv...@brusic.com) wrote: > > > > > It's only been around for 3.5 years: [https://github.com/](https://github.com/) > > > elasticsearch/elasticsearch/issues/623 🙂 > > > > > > I should clarify part of my previous statement. > > > > > > _"By default, the ongoing recovery is not cancelled when the missing > > > node rejoins the cluster. You can change the gateway settings [2] to > > > control when recovery kicks in."_ > > > > > > What I meant to say is that an ongoing recovery is never cancelled once > > > it has commenced, no matter what settings. By default, recovery happens > > > immediately, but can be changed with the gateway settings. > > > > > > -- > > > Ivan > > > > > > On Thu, Jul 10, 2014 at 1:48 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) [joerg...@gmail.com](mailto:joerg...@gmail.com) > > > wrote: > > > > > > > Indeed, auto\_expand\_replicas "all" triggers an update cluster > > > > settings action each time a node is added. > > > > > > > > Still blown by the many settings Elasticsearch provides. Feeling small. > > > > Homework: collecting a gist textfile of all ES 1.2 settings. > > > > > > > > Jörg > > > > > > > > On Thu, Jul 10, 2014 at 9:57 PM, Ivan Brusic [iv...@brusic.com](mailto:iv...@brusic.com) wrote: > > > > > > > > > Sticking to your use case, you might want to use > > > > > the auto\_expand\_replicas setting to "all" [1]: Never used it, but it sounds > > > > > what you are looking for. > > > > > > > > > > By default, the ongoing recovery is not cancelled when the missing > > > > > node rejoins the cluster. You can change the gateway settings [2] to > > > > > control when recovery kicks in. > > > > > > > > > > [1] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/) > > > > > reference/current/indices-update-settings.html > > > > > [2] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/) > > > > > reference/current/modules-gateway.html > > > > > > > > > > Cheers, > > > > > > > > > > Ivan > > > > > > > > > > On Thu, Jul 10, 2014 at 12:39 PM, Gonçalo Luiz [goncal...@gmail.com](mailto:goncal...@gmail.com) > > > > > wrote: > > > > > > > > > > > I get it know. > > > > > > > > > > > > I agree that setting the number of replicas is connected to the > > > > > > deployment reality in each case and it's derived variables and thus there > > > > > > is no one formula to fit all cases (it would't be a setting in that case). > > > > > > > > > > > > What I was trying to cover was the theoretical / extreme case where > > > > > > any node may fail at any time and what is the best way to go to minimize > > > > > > the chance of losing data. Also, in the case you want to scale down the > > > > > > installation (pottentially down to one node) without having to worry about > > > > > > selecting nodes that hold different replicated shards is an example that > > > > > > can beneffit from such configuration. > > > > > > > > > > > > I'm however not clear yet on what happens when a node goes down > > > > > > (triggering extra replication amongst the survivors) and then comes up > > > > > > again. Is the ongoing replication cancelled and the returning node brought > > > > > > up to date? > > > > > > > > > > > > Thanks for your valuable input. > > > > > > > > > > > > G. > > > > > > On 10 Jul 2014 18:07, "[joerg...@gmail.com](mailto:joerg...@gmail.com)" [joerg...@gmail.com](mailto:joerg...@gmail.com) > > > > > > wrote: > > > > > > > > > > > > > All I say is that it depends on the probability of the event of > > > > > > > three nodes failing simultaneously, not on the total number of nodes having > > > > > > > a replica. You can even have 5 nodes and the probability of the event of 4 > > > > > > > nodes failing simultaneously, and so on. > > > > > > > > > > > > > > As an illustration, suppose you have a data center with two > > > > > > > independent electric circuits and the probability of failure corresponds > > > > > > > with power outage, then it is enough to distribute nodes equally over > > > > > > > servers using the two independent power lines in the racks. If one electric > > > > > > > circuit (plus UPS) fails, half of the nodes go down. With replica level 1, > > > > > > > ES cluster will keep all the data. There is no need to set replica level > > > > > > > equal to node number. > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > On Thu, Jul 10, 2014 at 8:55 AM, Gonçalo Luiz [goncal...@gmail.com](mailto:goncal...@gmail.com) > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Joe, > > > > > > > > > > > > > > > > Thanks for your reply. > > > > > > > > On this thougth: > > > > > > > > > > > > > > > > " > > > > > > > > From my view your idea of better fault tolerance does not make much > > > > > > > > sense. The replica number is a statistical entity that is related to the > > > > > > > > probability of faults. The higher the replica, the higher the probability > > > > > > > > of surviving faults. There is no correlation to the total number of nodes > > > > > > > > in a cluster to ensure better fault tolerance. The fault tolerance depends > > > > > > > > on the probability of a node failure." > > > > > > > > > > > > > > > > I'm not getting it. If we have 4 nodes with 2 replicas it means > > > > > > > > that 3 of the nodes will have data of a given index (assuming 0 shards to > > > > > > > > ease the discussion), ritght? If those three nodes fail simultaneously the > > > > > > > > 4th will have no way of grabbing a copy and data will be lost forever. > > > > > > > > However if nr of replicas is 3, the 4th would be able to keep serving the > > > > > > > > requesrs and eventually handover a copy to a new node joining the cluster. > > > > > > > > How does this not help fault tolerance? I'm I missing something? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > G. > > > > > > > > On 10 Jul 2014 00:21, "[joerg...@gmail.com](mailto:joerg...@gmail.com)" [joerg...@gmail.com](mailto:joerg...@gmail.com) > > > > > > > > wrote: > > > > > > > > > > > > > > > > > 1. You can set replica number at index creation time or by cluster > > > > > > > > > update settings action org.elasticsearch. > > > > > > > > > action.admin.cluster.settings.ClusterUpdateSettingsAction > > > > > > > > > > > > > > > > > > 2. You will get an index with lower replica number 🙂 > > > > > > > > > > > > > > > > > > 3. Yes. Quick code example: > > > > > > > > > > > > > > > > > > 4. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > > > > > > > > > > > From my view your idea of better fault tolerance does not make > > > > > > > > > much sense. The replica number is a statistical entity that is related to > > > > > > > > > the probability of faults. The higher the replica, the higher the > > > > > > > > > probability of surviving faults. There is no correlation to the total > > > > > > > > > number of nodes in a cluster to ensure better fault tolerance. The fault > > > > > > > > > tolerance depends on the probability of a node failure. > > > > > > > > > > > > > > > > > > From the viewpoint of balancing load, it makes much sense. When > > > > > > > > > setting replica number to the number of nodes, the cluster can balance > > > > > > > > > search requests to all nodes which is optimal. > > > > > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > > > > > On Wed, Jul 9, 2014 at 11:57 PM, [goncal...@gmail.com](mailto:goncal...@gmail.com) wrote: > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > I'm considering using elasticsearch as a repository for a PoC I'm > > > > > > > > > > currently developing. > > > > > > > > > > > > > > > > > > > > This PoC models an application that needs durability but not > > > > > > > > > > isolability, so I'm fine with the eventual consistency of reads against the > > > > > > > > > > most recent writes. > > > > > > > > > > > > > > > > > > > > As durability is paramount (we can't affort to lose the data > > > > > > > > > > unless 100% of the nodes die) I've been exploring the option of setting > > > > > > > > > > every shard to have N replicas where N is the number of nodes in the > > > > > > > > > > cluster. > > > > > > > > > > > > > > > > > > > > From what I've read so far it is possible to dynamically set the > > > > > > > > > > number or replicas which triggers a replication throttled replication > > > > > > > > > > process. > > > > > > > > > > > > > > > > > > > > I would like to have some help on the following steps (I'm > > > > > > > > > > running ES in embedded mode in a Java application): > > > > > > > > > > > > > > > > > > > > 1 - How can I set the number or replicas using the native Java > > > > > > > > > > client ? > > > > > > > > > > 2 - What happens if a node dies and the number of replicas is > > > > > > > > > > lowered to the number of surviving ones? > > > > > > > > > > 3 - Is it possible, from a participating node, to access the list > > > > > > > > > > of nodes in the cluster so I can use their count to set the number of > > > > > > > > > > replicas (step 1) ? > > > > > > > > > > 4 - is it possible to hook a callback to the event of a node > > > > > > > > > > joining or leaving the cluster ? > > > > > > > > > > > > > > > > > > > > I envisioning the following mechanism: > > > > > > > > > > > > > > > > > > > > a) - Start with one node, a given number of shards and 1 replica > > > > > > > > > > b)- Each time a node joins I adjust the number or replicas to > > > > > > > > > > match the new node count. In this case, there would be 2 replicas > > > > > > > > > > c) - An arbitrary number of nodes might be added and I'd execute > > > > > > > > > > step b) accordingly > > > > > > > > > > d) - At any time a node might leave the cluster and thus I need > > > > > > > > > > to lower the number or replicas to the new node count (I assume that the > > > > > > > > > > cluster would go ahead and proceed to compensate the lost replica by asking > > > > > > > > > > an existing node to hold 2 replicas instead of one; is this stopped by > > > > > > > > > > lowering the number or replicas?) > > > > > > > > > > > > > > > > > > > > The ultimate goal is to make sure no data is loss unless 100% of > > > > > > > > > > the nodes die before a new one can acquire a full replica. > > > > > > > > > > > > > > > > > > > > Is this doable? Does this make sense at all ? > > > > > > > > > > > > > > > > > > > > For the time being, I'm not worried about lack of disk space or > > > > > > > > > > bandwidth as I'm still in the very early days of the PoC. > > > > > > > > > > > > > > > > > > > > Thank you very much for all your work and help. > > > > > > > > > > > > > > > > > > > > Gonçalo > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > You received this message because you are subscribed to the > > > > > > > > > > Google Groups "elasticsearch" group. > > > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > > > To view this discussion on the web visit > > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-](https://groups.google.com/d/msgid/elasticsearch/276418fa-) > > > > > > > > > > 812f-4af5-94a0-7362f5ba7931%[40googlegroups.com](http://40googlegroups.com) > > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/276418fa-812f-4af5-94a0-7362f5ba7931%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > > > > > -- > > > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > > > > topic/elasticsearch/hPvVz20v6YY/unsubscribe. > > > > > > > > > To unsubscribe from this group and all its topics, send an email > > > > > > > > > to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > > To view this discussion on the web visit > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/](https://groups.google.com/d/msgid/elasticsearch/) > > > > > > > > > CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail. > > > > > > > > > [gmail.com](http://gmail.com) > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGuUAJQYkkRtTU1H90MKpRg46dM1T2PzcP2Mfk1X8mbfA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > > > > . > > > > > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_) > > > > > > > > xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%[40mail.gmail.com](http://40mail.gmail.com) > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B\_xoC2dXVXVzaXFxV70Q\_XGs%2BrXoWqnSu\_SrVvGJw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09qtY%2B_xoC2dXVXVzaXFxV70Q_XGs%2BrXoWqnSu_SrVvGJw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > > > . > > > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > > topic/elasticsearch/hPvVz20v6YY/unsubscribe. > > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%25) > > > > > > > 2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%[40mail.gmail.com](http://40mail.gmail.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHv1Ab6ite%2BHfaVh9ti2ea69Bh0ASjkCM8vE6%2Bn7j3PgQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_ > > > > > > S0LWt3tRJHNihWg%[40mail.gmail.com](http://40mail.gmail.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB\_S0LWt3tRJHNihWg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAMrm09r7EU4ZMy4%2BfVZ3SXmTPbh4YDsJgB_S0LWt3tRJHNihWg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ > > > > > ROLsSTg%[40mail.gmail.com](http://40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC\_W3mwCBKBOBjt95tRwj31J9Zq5XP\_ROLsSTg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBhiw5DqC_W3mwCBKBOBjt95tRwj31J9Zq5XP_ROLsSTg%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_ > > > > GFns4QsGCML5eS5LT6yiEDf6Q%[40mail.gmail.com](http://40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd\_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcroqUqU7an5B8bHQd_GFns4QsGCML5eS5LT6yiEDf6Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > topic/elasticsearch/hPvVz20v6YY/unsubscribe. > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/CALY%3DcQA\_LtOdS-6Ht\_DR57P%2BXkLWp5%3DV5Dz%2BVh2\_ > > > cMkgy6kDSw%[40mail.gmail.com](http://40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA\_LtOdS-6Ht\_DR57P%2BXkLWp5%3DV5Dz%2BVh2\_cMkgy6kDSw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQA_LtOdS-6Ht_DR57P%2BXkLWp5%3DV5Dz%2BVh2_cMkgy6kDSw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAMrm09pWmsimVqxRVYf0iSozP\_QMrU-\_p%3DUODWd9CS4O\_Ti5Wg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAMrm09pWmsimVqxRVYf0iSozP_QMrU-_p%3DUODWd9CS4O_Ti5Wg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 11, 2014, 6:41pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/18 "2014-07-11T18:41:48Z")
They work both ways. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFLPD9NH%2BTf4aoHCTPdZ4EQ2CnDB75VaeZv%3D%3DV%2BtUUBVA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFLPD9NH%2BTf4aoHCTPdZ4EQ2CnDB75VaeZv%3D%3DV%2BtUUBVA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![ElasticSearch\_Users\_](https://avatars.discourse-cdn.com/v4/letter/e/b4bc9f/32.png) [@ElasticSearch\_Users\_](https://discuss.elastic.co/u/ElasticSearch_Users_)\ **Post date:** [August 22, 2014, 8:08am UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/19 "2014-08-22T08:08:53Z")
Hi Jorg, Can you please give a server-side or client-side example of using CLusterStateListener? Do I have to use a plugin. if so, which module do I register/override? If not, do I have to use a Node Client (not a TransportClient), and retrieve the ClusterService somehow and then register? Thanks Sandeep On Thursday, 10 July 2014 22:25:51 UTC+5:30, Jörg Prante wrote: > On the client side, you can't use cluster state listener, it is for nodes > that have access to a local copy of the master cluster state. Clients must > execute an action to ask for cluster state, and with the current transport > request/response cycle, they must poll for new events ... > > Jörg > > On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic \<[iv...@brusic.com](mailto:iv...@brusic.com) > \\> wrote: > > > Jörg, have you actually implemented your own ClusterStateListener? I > > never had much success. Tried using that interface or > > even PublishClusterStateAction.NewClusterStateListener, but either I could > > not configure successfully the module (the former) or received no events > > (the latter). Implemented on the client side, not as a plugin. > > > > Cheers, > > > > Ivan > > > > On Wed, Jul 9, 2014 at 4:21 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) \ \< > > [joerg...@gmail.com](mailto:joerg...@gmail.com) \\> wrote: > > > > > 1. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [August 23, 2014, 3:57pm UTC](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563/20 "2014-08-23T15:57:08Z")
As mentioned by Jörg early on in the thread: "On the client side, you can't use cluster state listener" I never bothered with a plugin version since the interaction would be too kludgy. -- Ivan On Fri, Aug 22, 2014 at 1:08 AM, 'Sandeep Ramesh Khanzode' via elasticsearch [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) wrote: > Hi Jorg, > > Can you please give a server-side or client-side example of using > CLusterStateListener? > Do I have to use a plugin. if so, which module do I register/override? > If not, do I have to use a Node Client (not a TransportClient), and > retrieve the ClusterService somehow and then register? > > Thanks > Sandeep > > On Thursday, 10 July 2014 22:25:51 UTC+5:30, Jörg Prante wrote: > > > On the client side, you can't use cluster state listener, it is for nodes > > that have access to a local copy of the master cluster state. Clients must > > execute an action to ask for cluster state, and with the current transport > > request/response cycle, they must poll for new events ... > > > > Jörg > > > > On Thu, Jul 10, 2014 at 6:38 PM, Ivan Brusic [iv...@brusic.com](mailto:iv...@brusic.com) wrote: > > > > > Jörg, have you actually implemented your own ClusterStateListener? I > > > never had much success. Tried using that interface or even > > > PublishClusterStateAction.NewClusterStateListener, but either I could > > > not configure successfully the module (the former) or received no events > > > (the latter). Implemented on the client side, not as a plugin. > > > > > > Cheers, > > > > > > Ivan > > > > > > On Wed, Jul 9, 2014 at 4:21 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) [joerg...@gmail.com](mailto:joerg...@gmail.com) > > > wrote: > > > > > > > 1. Yes. Use org.elasticsearch.cluster.ClusterStateListener > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i- > > > > sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%[40mail.gmail.com](http://40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW\_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBB%3DW_qG9E7i-sEc6HZeMskxKgbqzaKgqzSQ26sjgT5%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/35f6b64e-3787-4891-a3a8-518dfd7638e3%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDo1uqMJPjqcx7T4DKiCERHVthEqWEVpwW6jUcKn5x\_3w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDo1uqMJPjqcx7T4DKiCERHVthEqWEVpwW6jUcKn5x_3w%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). [Next page](https://discuss.elastic.co/t/elastic-search-dynamic-number-of-replicas-from-java-api/18563.md?page=2)