# Rolling Restart an Elasticsearch Cluster with Ansible **URL:** \ **Category:** Elasticsearch\ **Created:** [September 17, 2014, 7:43pm UTC](https://discuss.elastic.co/t/rolling-restart-an-elasticsearch-cluster-with-ansible/19851 "2014-09-17T19:43:01Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![labrown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/labrown/32/464_2.png) [@labrown](https://discuss.elastic.co/u/labrown)\ **Post date:** [September 17, 2014, 7:43pm UTC](https://discuss.elastic.co/t/rolling-restart-an-elasticsearch-cluster-with-ansible/19851/1 "2014-09-17T19:43:01Z")
I've come up with what I think is a safe way to rolling restart an Elasticsearch cluster using Ansible handlers. Why is this needed? Even if you use a serial setting to limit the number of nodes processed at one time, Ansible will restart elasticsearch nodes and continue processing as soon as the elasticsearch service restart reports itself complete. This pushes the cluster into a red state due to muliple data nodes being restarted at once, and can cause performance problems. Solution: Perform a rolling restart of each elasticsearch node and wait for the cluster to stabilize before continuing processing nodes. This set of chained handlers restarts each node while keeping the cluster from thrashing on reallocating shards during the process. A gist of the handlers/main.yml file for my Elasticsearch role is at > I welcome any comments and/or suggestions. --[Lance] -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ed35d8af7b9fb038b5b821b855ad75ba%40webmail.bearcircle.net](https://groups.google.com/d/msgid/elasticsearch/ed35d8af7b9fb038b5b821b855ad75ba%40webmail.bearcircle.net). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [September 18, 2014, 5:43am UTC](https://discuss.elastic.co/t/rolling-restart-an-elasticsearch-cluster-with-ansible/19851/2 "2014-09-18T05:43:51Z")
Hello Lance, This looks really nice and useful, thanks for sharing! When yous end a cluster health command, you can also tell it to wait\_for\_status=green or wait\_for\_status=yellow with a specified timeout: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. This seems a nicer approach than to repeat the request and verify the outcome from Ansible. But then again, if you just restarted a node, you don't know when the node responds to requests at all. So waiting for green might work on the last step, but maybe not when you restart each node (to wait for yellow, since you disabled shard allocation), unless the init script ensures ES is responsive when it returns successful. Best regards, Radu On Wednesday, September 17, 2014 10:43:10 PM UTC+3, Lance A. Brown wrote: > I've come up with what I think is a safe way to rolling restart an > Elasticsearch cluster using Ansible handlers. > > Why is this needed? > > Even if you use a serial setting to limit the number of nodes processed > at one time, Ansible will restart elasticsearch nodes and continue > processing as soon as the elasticsearch service restart reports itself > complete. This pushes the cluster into a red state due to muliple data > nodes being restarted at once, and can cause performance problems. > > Solution: > > Perform a rolling restart of each elasticsearch node and wait for the > cluster to stabilize before continuing processing nodes. This set of > chained handlers restarts each node while keeping the cluster from > thrashing on reallocating shards during the process. > > A gist of the handlers/main.yml file for my Elasticsearch role is at > [Ansible rolling restart of Elasticsearch Cluster · GitHub](https://gist.github.com/labrown/5341ebec47bfba6dd7d4) > > I welcome any comments and/or suggestions. > > --[Lance] -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eadcf01e-212b-4a5c-8b2d-6171d8085ebd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eadcf01e-212b-4a5c-8b2d-6171d8085ebd%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:01am UTC](https://discuss.elastic.co/t/rolling-restart-an-elasticsearch-cluster-with-ansible/19851/3 "2017-07-06T01:01:27Z")