# Automatic partial shutdown of cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [August 14, 2014, 4:23am UTC](https://discuss.elastic.co/t/automatic-partial-shutdown-of-cluster/19275 "2014-08-14T04:23:19Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![krishnan](https://avatars.discourse-cdn.com/v4/letter/k/a698b9/32.png) [@krishnan](https://discuss.elastic.co/u/krishnan)\ **Post date:** [August 14, 2014, 4:23am UTC](https://discuss.elastic.co/t/automatic-partial-shutdown-of-cluster/19275/1 "2014-08-14T04:23:19Z")
Hi All, I have cluster of one master only nodes and 7 data only nodes. Some nodes are getting shutdown automatically. Could you please advise what is the possible cause of this behaviour or how do I investigate the issue. For you perusal the log entry is below : Data node logs : [2014-08-13 17:10:22,882][INFO][action.admin.cluster.node.shutdown] [search7] shutting down in [200ms] [2014-08-13 17:10:23,102][INFO][action.admin.cluster.node.shutdown] [search7] initiating requested shutdown... [2014-08-13 17:10:23,106][INFO][node] [search7] stopping ... [2014-08-13 17:10:52,290][INFO][node] [search7] stopped [2014-08-13 17:10:52,290][INFO][node] [search7] closing ... [2014-08-13 17:10:52,323][INFO][node] [search7] closed Server node logs : [2014-08-13 17:10:21,879][INFO][action.admin.cluster.node.shutdown] [search1] [partial\_cluster\_shutdown]: requested, shutting down [[QowbqobdTyqrMZ00EVSvbw]] in [1s] [2014-08-13 17:10:21,991][INFO][action.admin.cluster.node.shutdown] [search1] [partial\_cluster\_shutdown]: requested, shutting down [[hRJliGmSSTuXX1Kmmjz94Q]] in [1s] [2014-08-13 17:10:22,206][INFO][action.admin.cluster.node.shutdown] [search1] [partial\_cluster\_shutdown]: requested, shutting down [[dqnFXTV9RoKeJpSy7zNqPA]] in [1s] [2014-08-13 17:10:22,901][INFO][action.admin.cluster.node.shutdown] [search1] [partial\_cluster\_shutdown]: done shutting down [[QowbqobdTyqrMZ00EVSvbw]] [2014-08-13 17:10:22,999][INFO][action.admin.cluster.node.shutdown] [search1] [partial\_cluster\_shutdown]: done shutting down [[hRJliGmSSTuXX1Kmmjz94Q]] [2014-08-13 17:10:23,290][INFO][action.admin.cluster.node.shutdown] [search1] [partial\_cluster\_shutdown]: done shutting down [[dqnFXTV9RoKeJpSy7zNqPA]] Thank You, Krishna -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 14, 2014, 4:32am UTC](https://discuss.elastic.co/t/automatic-partial-shutdown-of-cluster/19275/2 "2014-08-14T04:32:31Z")
Something is shutting your node down using the API. You might want to look at > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 14 August 2014 14:23, Krishnan [krishna.dave@gmail.com](mailto:krishna.dave@gmail.com) wrote: > Hi All, > > I have cluster of one master only nodes and 7 data only nodes. Some nodes > are getting shutdown automatically. Could you please advise what is the > possible cause of this behaviour or how do I investigate the issue. > > For you perusal the log entry is below : > > Data node logs : > > [2014-08-13 17:10:22,882][INFO][action.admin.cluster.node.shutdown] > [search7] shutting down in [200ms] > [2014-08-13 17:10:23,102][INFO][action.admin.cluster.node.shutdown] > [search7] initiating requested shutdown... > [2014-08-13 17:10:23,106][INFO][node] [search7] > stopping ... > [2014-08-13 17:10:52,290][INFO][node] [search7] > stopped > [2014-08-13 17:10:52,290][INFO][node] [search7] > closing ... > [2014-08-13 17:10:52,323][INFO][node] [search7] > closed > > Server node logs : > [2014-08-13 17:10:21,879][INFO][action.admin.cluster.node.shutdown] > [search1] [partial\_cluster\_shutdown]: requested, shutting down > [[QowbqobdTyqrMZ00EVSvbw]] in [1s] > [2014-08-13 17:10:21,991][INFO][action.admin.cluster.node.shutdown] > [search1] [partial\_cluster\_shutdown]: requested, shutting down > [[hRJliGmSSTuXX1Kmmjz94Q]] in [1s] > [2014-08-13 17:10:22,206][INFO][action.admin.cluster.node.shutdown] > [search1] [partial\_cluster\_shutdown]: requested, shutting down > [[dqnFXTV9RoKeJpSy7zNqPA]] in [1s] > [2014-08-13 17:10:22,901][INFO][action.admin.cluster.node.shutdown] > [search1] [partial\_cluster\_shutdown]: done shutting down > [[QowbqobdTyqrMZ00EVSvbw]] > [2014-08-13 17:10:22,999][INFO][action.admin.cluster.node.shutdown] > [search1] [partial\_cluster\_shutdown]: done shutting down > [[hRJliGmSSTuXX1Kmmjz94Q]] > [2014-08-13 17:10:23,290][INFO][action.admin.cluster.node.shutdown] > [search1] [partial\_cluster\_shutdown]: done shutting down > [[dqnFXTV9RoKeJpSy7zNqPA]] > > Thank You, > Krishna > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624beY-NaK15DqeBDM3N5Ayfe%2BJZMTpBdkG3\_Z6NNcjAkdQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624beY-NaK15DqeBDM3N5Ayfe%2BJZMTpBdkG3_Z6NNcjAkdQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![krishnan](https://avatars.discourse-cdn.com/v4/letter/k/a698b9/32.png) [@krishnan](https://discuss.elastic.co/u/krishnan)\ **Post date:** [August 14, 2014, 8:04am UTC](https://discuss.elastic.co/t/automatic-partial-shutdown-of-cluster/19275/3 "2014-08-14T08:04:57Z")
Thank you so much for the suggestion, Walkom. Now how do I find out what is shutting down my nodes? I checked /var/logs, did not find any thing suspicious. is this related to out of memory system states. Regards, Krishna On Thursday, August 14, 2014 10:03:10 AM UTC+5:30, Mark Walkom wrote: > Something is shutting your node down using the API. > > You might want to look at > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-shutdown.html#_disable_shutdown) > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \ > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 14 August 2014 14:23, Krishnan \<[krishn...@gmail.com](mailto:krishn...@gmail.com) \\> > wrote: > > > Hi All, > > > > I have cluster of one master only nodes and 7 data only nodes. Some nodes > > are getting shutdown automatically. Could you please advise what is the > > possible cause of this behaviour or how do I investigate the issue. > > > > For you perusal the log entry is below : > > > > Data node logs : > > > > [2014-08-13 17:10:22,882][INFO][action.admin.cluster.node.shutdown] > > [search7] shutting down in [200ms] > > [2014-08-13 17:10:23,102][INFO][action.admin.cluster.node.shutdown] > > [search7] initiating requested shutdown... > > [2014-08-13 17:10:23,106][INFO][node] [search7] > > stopping ... > > [2014-08-13 17:10:52,290][INFO][node] [search7] > > stopped > > [2014-08-13 17:10:52,290][INFO][node] [search7] > > closing ... > > [2014-08-13 17:10:52,323][INFO][node] [search7] > > closed > > > > Server node logs : > > [2014-08-13 17:10:21,879][INFO][action.admin.cluster.node.shutdown] > > [search1] [partial\_cluster\_shutdown]: requested, shutting down > > [[QowbqobdTyqrMZ00EVSvbw]] in [1s] > > [2014-08-13 17:10:21,991][INFO][action.admin.cluster.node.shutdown] > > [search1] [partial\_cluster\_shutdown]: requested, shutting down > > [[hRJliGmSSTuXX1Kmmjz94Q]] in [1s] > > [2014-08-13 17:10:22,206][INFO][action.admin.cluster.node.shutdown] > > [search1] [partial\_cluster\_shutdown]: requested, shutting down > > [[dqnFXTV9RoKeJpSy7zNqPA]] in [1s] > > [2014-08-13 17:10:22,901][INFO][action.admin.cluster.node.shutdown] > > [search1] [partial\_cluster\_shutdown]: done shutting down > > [[QowbqobdTyqrMZ00EVSvbw]] > > [2014-08-13 17:10:22,999][INFO][action.admin.cluster.node.shutdown] > > [search1] [partial\_cluster\_shutdown]: done shutting down > > [[hRJliGmSSTuXX1Kmmjz94Q]] > > [2014-08-13 17:10:23,290][INFO][action.admin.cluster.node.shutdown] > > [search1] [partial\_cluster\_shutdown]: done shutting down > > [[dqnFXTV9RoKeJpSy7zNqPA]] > > > > Thank You, > > Krishna > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1cf8f1ca-6388-4ffa-b4aa-805fc4254b8e%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b049afa9-582f-4ffe-bf0e-9150a816611b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b049afa9-582f-4ffe-bf0e-9150a816611b%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:08am UTC](https://discuss.elastic.co/t/automatic-partial-shutdown-of-cluster/19275/4 "2017-07-06T01:08:40Z")