# What is the proper way to restart node without clients noticing? **URL:** \ **Category:** Elasticsearch\ **Created:** [June 18, 2013, 1:06pm UTC](https://discuss.elastic.co/t/what-is-the-proper-way-to-restart-node-without-clients-noticing/12459 "2013-06-18T13:06:49Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Maciej\_Dziardziel](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@Maciej\_Dziardziel](https://discuss.elastic.co/u/Maciej_Dziardziel)\ **Post date:** [June 18, 2013, 1:06pm UTC](https://discuss.elastic.co/t/what-is-the-proper-way-to-restart-node-without-clients-noticing/12459/1 "2013-06-18T13:06:49Z")
Hi I have a cluster of few es nodes. Due to changes in java scripts, or maintenance, sometimes i need to restart them. What is the proper way of doing it without clients noticing? I was thinking about setting up additional nodes with node.data set to false. Clients would connect to those non-data nodes. Now i would only need a way to say to es cluster "i'm going to shut down this node in 10 minutes, so let it finish processing its requests, and redirect new ones to other nodes". Is it possibe? -- Maciej Dziardziel ([fiedzia@gmail.com](mailto:fiedzia@gmail.com)) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [June 19, 2013, 2:14pm UTC](https://discuss.elastic.co/t/what-is-the-proper-way-to-restart-node-without-clients-noticing/12459/2 "2013-06-19T14:14:00Z")
If your replication values and quorum settings are correct, you can disable allocation and simply restart the nodes. Assuming the shards are correctly distributed, the other nodes will contain all the search data. It helps to disable indexing as well and flush the transaction logs so that you can return the the original state after the nodes rejoin and allocation is reenabled. Cheers, Ivan On Tue, Jun 18, 2013 at 6:06 AM, Maciej Dziardziel [fiedzia@gmail.com](mailto:fiedzia@gmail.com)wrote: > Hi > > I have a cluster of few es nodes. Due to changes in java scripts, or > maintenance, sometimes i need to restart them. > What is the proper way of doing it without clients noticing? > > I was thinking about setting up additional nodes with node.data set to > false. Clients would connect to those non-data nodes. > Now i would only need a way to say to es cluster "i'm going to shut down > this node in 10 minutes, so let it finish processing its requests, > and redirect new ones to other nodes". Is it possibe? > > -- > Maciej Dziardziel ([fiedzia@gmail.com](mailto:fiedzia@gmail.com)) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Maciej\_Dziardziel](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@Maciej\_Dziardziel](https://discuss.elastic.co/u/Maciej_Dziardziel)\ **Post date:** [June 19, 2013, 10:08pm UTC](https://discuss.elastic.co/t/what-is-the-proper-way-to-restart-node-without-clients-noticing/12459/3 "2013-06-19T22:08:02Z")
Hi Thanks for your answer. Yes, other nodes will contain data and new requests can be handled by them. The problem is that: 1. Every request from a client connected to the node i am going to restart will fail when i do that. 2. It will take time and some failed requests for a client to realize that a node is down and it should speak to others. That's the problems i was trying to avoid. Since i can't find anything useful in ES documentation, i worked around it with a load balancer that allows me to gracefully remove nodes (without breaking open connections). It would be handy to have such feature in ES by default though. On Wednesday, June 19, 2013 3:14:00 PM UTC+1, Ivan Brusic wrote: > If your replication values and quorum settings are correct, you can > disable allocation and simply restart the nodes. Assuming the shards are > correctly distributed, the other nodes will contain all the search data. It > helps to disable indexing as well and flush the transaction logs so that > you can return the the original state after the nodes rejoin and allocation > is reenabled. > > Cheers, > > Ivan > > On Tue, Jun 18, 2013 at 6:06 AM, Maciej Dziardziel \<[fie...@gmail.com](mailto:fie...@gmail.com)\ > > > wrote: > > > Hi > > > > I have a cluster of few es nodes. Due to changes in java scripts, or > > maintenance, sometimes i need to restart them. > > What is the proper way of doing it without clients noticing? > > > > I was thinking about setting up additional nodes with node.data set to > > false. Clients would connect to those non-data nodes. > > Now i would only need a way to say to es cluster "i'm going to shut down > > this node in 10 minutes, so let it finish processing its requests, > > and redirect new ones to other nodes". Is it possibe? > > > > -- > > Maciej Dziardziel ([fie...@gmail.com](mailto:fie...@gmail.com) \) > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [June 19, 2013, 11:19pm UTC](https://discuss.elastic.co/t/what-is-the-proper-way-to-restart-node-without-clients-noticing/12459/4 "2013-06-19T23:19:18Z")
Are you using the TransportClient or the REST client? The TransportClient should handle most of these issues. Can't really help if you are using HTTP. Cheers, Ivan On Wed, Jun 19, 2013 at 3:08 PM, Maciej Dziardziel [fiedzia@gmail.com](mailto:fiedzia@gmail.com)wrote: > Hi > > Thanks for your answer. Yes, other nodes will contain data and new > requests can be handled by them. The problem is that: > > 1. Every request from a client connected to the node i am going to restart > will fail when i do that. > 2. It will take time and some failed requests for a client to realize that > a node is down and it should speak to others. > > That's the problems i was trying to avoid. > > Since i can't find anything useful in ES documentation, i worked around it > with a load balancer that allows me to gracefully remove nodes > (without breaking open connections). It would be handy to have such > feature in ES by default though. > > On Wednesday, June 19, 2013 3:14:00 PM UTC+1, Ivan Brusic wrote: > > > If your replication values and quorum settings are correct, you can > > disable allocation and simply restart the nodes. Assuming the shards are > > correctly distributed, the other nodes will contain all the search data. It > > helps to disable indexing as well and flush the transaction logs so that > > you can return the the original state after the nodes rejoin and allocation > > is reenabled. > > > > Cheers, > > > > Ivan > > > > On Tue, Jun 18, 2013 at 6:06 AM, Maciej Dziardziel [fie...@gmail.com](mailto:fie...@gmail.com)wrote: > > > > > Hi > > > > > > I have a cluster of few es nodes. Due to changes in java scripts, or > > > maintenance, sometimes i need to restart them. > > > What is the proper way of doing it without clients noticing? > > > > > > I was thinking about setting up additional nodes with node.data set to > > > false. Clients would connect to those non-data nodes. > > > Now i would only need a way to say to es cluster "i'm going to shut > > > down this node in 10 minutes, so let it finish processing its requests, > > > and redirect new ones to other nodes". Is it possibe? > > > > > > -- > > > Maciej Dziardziel ([fie...@gmail.com](mailto:fie...@gmail.com)) > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out) > > > . > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![sujoysett](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sujoysett](https://discuss.elastic.co/u/sujoysett)\ **Post date:** [June 20, 2013, 6:06am UTC](https://discuss.elastic.co/t/what-is-the-proper-way-to-restart-node-without-clients-noticing/12459/5 "2013-06-20T06:06:43Z")
Hi, Once used [GitHub - exalead/commons-httpclient-failover: Load-balancing and failover support for Apache HttpClient 3.X](https://github.com/exalead/commons-httpclient-failover) with elasticsearch multiple nodes in a cluster. That was long back though, not sure whether its still relevant or not. - Sujoy. On Thursday, June 20, 2013 3:38:02 AM UTC+5:30, Maciej Dziardziel wrote: > Hi > > Thanks for your answer. Yes, other nodes will contain data and new > requests can be handled by them. The problem is that: > > 1. Every request from a client connected to the node i am going to restart > will fail when i do that. > 2. It will take time and some failed requests for a client to realize that > a node is down and it should speak to others. > > That's the problems i was trying to avoid. > > Since i can't find anything useful in ES documentation, i worked around it > with a load balancer that allows me to gracefully remove nodes > (without breaking open connections). It would be handy to have such > feature in ES by default though. > > On Wednesday, June 19, 2013 3:14:00 PM UTC+1, Ivan Brusic wrote: > > > If your replication values and quorum settings are correct, you can > > disable allocation and simply restart the nodes. Assuming the shards are > > correctly distributed, the other nodes will contain all the search data. It > > helps to disable indexing as well and flush the transaction logs so that > > you can return the the original state after the nodes rejoin and allocation > > is reenabled. > > > > Cheers, > > > > Ivan > > > > On Tue, Jun 18, 2013 at 6:06 AM, Maciej Dziardziel [fie...@gmail.com](mailto:fie...@gmail.com)wrote: > > > > > Hi > > > > > > I have a cluster of few es nodes. Due to changes in java scripts, or > > > maintenance, sometimes i need to restart them. > > > What is the proper way of doing it without clients noticing? > > > > > > I was thinking about setting up additional nodes with node.data set to > > > false. Clients would connect to those non-data nodes. > > > Now i would only need a way to say to es cluster "i'm going to shut > > > down this node in 10 minutes, so let it finish processing its requests, > > > and redirect new ones to other nodes". Is it possibe? > > > > > > -- > > > Maciej Dziardziel ([fie...@gmail.com](mailto:fie...@gmail.com)) > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:30am UTC](https://discuss.elastic.co/t/what-is-the-proper-way-to-restart-node-without-clients-noticing/12459/6 "2017-07-06T02:30:18Z")