# ES 6.2.4 How to find out which nodes are offline

**URL:** <https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339>\
**Category:** Elasticsearch\
**Created:** [June 3, 2018, 10:03pm UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339 "2018-06-03T22:03:01Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\
**Post date:** [June 3, 2018, 10:03pm UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339/1 "2018-06-03T22:03:01Z")

</div>

Hi Team,

What is the recommended way to find out which nodes in the cluster are offline using the rest API?

In Kibana monitoring I can see which nodes are offline but I am not sure how Kibana actually determines that - does it do a comparison between a previously known list of nodes?

Cheers,

---

<div class="post-metadata">

**Author:** ![azhar](https://avatars.discourse-cdn.com/v4/letter/a/74df32/32.png) [@azhar](https://discuss.elastic.co/u/azhar)\
**Post date:** [June 3, 2018, 10:24pm UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339/2 "2018-06-03T22:24:25Z")

</div>

Hi,

You can find the number of active nodes by querying `GET /_nodes/stats` in the Kibana console or from the following rest end point - [http://ip:9200/\_nodes/stats](http://ip:9200/_nodes/stats)

---

<div class="post-metadata">

**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\
**Post date:** [June 3, 2018, 10:36pm UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339/3 "2018-06-03T22:36:12Z")

</div>

Thanks, my impression was `GET _nodes/stats` only shows nodes that are up and not those that are down? Also, from the JSON returned, do I have to look at the `processs.timestamp` key and figure out when it was up last - how do I determine that the node is down from the JSON?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 4, 2018, 12:46am UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339/4 "2018-06-04T00:46:59Z")

</div>

That's true.  
I believe that Monitoring stores the fact that a node has been seen then if the node does not report statistics after a while to the monitoring cluster then its considered as down.

I did not check the code though.

---

<div class="post-metadata">

**Author:** ![Bernt\_Rostad](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@Bernt\_Rostad](https://discuss.elastic.co/u/Bernt_Rostad)\
**Post date:** [June 4, 2018, 9:44am UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339/5 "2018-06-04T09:44:39Z")

</div>

I ran into the same problem a few years ago: There is nothing in `GET _nodes/stats` telling you if a node has dropped out of the cluster. And Kibana monitoring will only show the node (in gray) for the duration of the selected Time Range (1 hour by default), after that it will be gone from Kibana surveillance too, until it rejoins the cluster.

My solution was to create a simple cluster deployment configuration, which I found useful for other things too (such as full cluster restarts), in which I list all the nodes that should be in a particular cluster. Then a monitoring script uses this configuration when checking `_nodes/stats` and reports if a node is missing. This requires a bit of scripting but is a straight forward job for a programmer.

---

<div class="post-metadata">

**Author:** ![ld\_pvl](https://avatars.discourse-cdn.com/v4/letter/l/cdc98d/32.png) [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\
**Post date:** [June 4, 2018, 10:11am UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339/6 "2018-06-04T10:11:14Z")

</div>

Thanks David and Bernt!

We have the same approach and have a config file as well that lists out all the nodes that are supposed to be running (with their settings/attributes) which I was planning to use to do the comparison with `GET _cat/nodes` (which also only shows members that are up).

Just wanted to check that I'm not going the more complicated way before I implement the above - thanks guys for giving me the confidence 😄 .

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 2, 2018, 10:11am UTC](https://discuss.elastic.co/t/es-6-2-4-how-to-find-out-which-nodes-are-offline/134339/7 "2018-07-02T10:11:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
