# One ES node not showing in Kibana Monitoring
**URL:** https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212
**Category:** Kibana
**Tags:** elastic-stack-monitoring
**Created:** [September 19, 2019, 1:47pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212 "2019-09-19T13:47:21Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [September 19, 2019, 1:47pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/1 "2019-09-19T13:47:21Z")
I recently upgraded 3 (of 9) nodes to new hardware. Since that time, one of the nodes (es02) is not visible in the Kibana "Monitoring" page. ALL nodes are visible via Elasticsearch Head plugin.
Version 6.8.3
Based on earlier report from another user (aparently abandoned by the user), I will attach the results requested by one of the ES developers for that ticket.
In the results below, the nodes are es01,02,03,5,06,07,08,09, and hd3. The node that is not displaying is es02; the other new machines that are configured the same way and are working fine are es01 and es03. Please note that the node appears to be otherwise working fine.
1. GET \_cat/indices?v
2. GET \_cat/nodes?v
3. GET \_cluster/settings?include\_defaults&filter\_path=_.xpack.monitoring._
4. POST .monitoring-es-\*/\_search
5. Could you post all xpack.monitoring.\* settings from one of your "good" node's elasticsearch.yml? Similarly, could you post all xpack.monitoring.\* settings from one of your "bad" node's elasticsearch.yml?
\*\* Please advise how/where to send the result files. I thought I could attach them here.
---
### Author:  [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)
#### Post date: [September 24, 2019, 2:19pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/2 "2019-09-24T14:19:21Z")
Hi @thealy
Could you please query the cluster where the monitoring data is stored with the following?
`GET /.monitoring-es*/_search?filter_path=hits.hits._source.node_stats.node_id`
You should get a list of node IDs. Could you please compare that list to what's returned by `GET /_nodes?filter_path=nodes.*.name` see if the node that's not appearing (es02) is present?
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [September 24, 2019, 2:37pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/3 "2019-09-24T14:37:32Z")
Hi Mike-
Thanks for your help. Here are the results:
GET /.monitoring-es\*/\_search?filter\_path=hits.hits.\_source.node\_stats.node\_id
indent preformatted text by 4 spaces
{
"hits" : {
"hits" : [
{
"\_source" : {
"node\_stats" : {
"node\_id" : "3PgfzMugQhihpx1eiHEfOQ"
}
}
}
]
}
}
GET /\_nodes?filter\_path=nodes.\*.name
{
"nodes" : {
"pelqvr1NSCWkRCQ0B0uonw" : {
"name" : "es02"
},
"L\_4bDstyTlO4ojV577QkvA" : {
"name" : "es07"
},
"3PgfzMugQhihpx1eiHEfOQ" : {
"name" : "hd3"
},
"U30RaA3LQFqTKCLvvBEYCA" : {
"name" : "es09"
},
"ny\_bTxplRwq1YUsnNDswPQ" : {
"name" : "es08"
},
"wbeNKPt7T\_WmYDzkeW0TMA" : {
"name" : "es03"
},
"MuCs6HxkT0W9IqKSz--Bpw" : {
"name" : "es05"
},
"QYoIq7mlQ9W4DpyymdfR2w" : {
"name" : "es06"
},
"IG8pEj3WRi-Ia\_seu2HVvg" : {
"name" : "es01"
}
}
}
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [September 27, 2019, 3:08pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/4 "2019-09-27T15:08:54Z")
Any thoughts Mike?
---
### Author:  [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)
#### Post date: [September 30, 2019, 7:28am UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/5 "2019-09-30T07:28:02Z")
@thealy Are you certain that `GET /.monitoring-es*/_search?filter_path=hits.hits._source.node_stats.node_id` was run from the **monitoring cluster**? The other nodes should be shown there, so this is a bit surprising.
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [October 8, 2019, 3:19pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/6 "2019-10-08T15:19:18Z")
I'm afraid I don't understand what you are asking. We have only one cluster, and the command was run there. The "missing" node (es02) is in the list.
{
"nodes" : {
"wbeNKPt7T\_WmYDzkeW0TMA" : {
"name" : "es03"
},
"QYoIq7mlQ9W4DpyymdfR2w" : {
"name" : "es06"
},
"pelqvr1NSCWkRCQ0B0uonw" : {
"name" : "es02"
},
"IG8pEj3WRi-Ia\_seu2HVvg" : {
"name" : "es01"
},
"L\_4bDstyTlO4ojV577QkvA" : {
"name" : "es07"
},
"ny\_bTxplRwq1YUsnNDswPQ" : {
"name" : "es08"
},
"MuCs6HxkT0W9IqKSz--Bpw" : {
"name" : "es05"
},
"3PgfzMugQhihpx1eiHEfOQ" : {
"name" : "hd3"
},
"U30RaA3LQFqTKCLvvBEYCA" : {
"name" : "es09"
}
}
}
---
### Author:  [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [October 8, 2019, 3:22pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/7 "2019-10-08T15:22:56Z")
Let's try a slightly different query.
```auto
GET .monitoring-es-*/_search?filter_path=aggregations.node_ids
{
"size": 0,
"query": {
"term": {
"type": {
"value": "node_stats"
}
}
},
"aggs": {
"node_ids": {
"terms": {
"field": "node_stats.node_id",
"size": 50
}
}
}
}
```
Do you see the `es02` showing up there? Lemme know and don't worry - we'll figure this out!
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [October 8, 2019, 4:03pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/8 "2019-10-08T16:03:07Z")
Yes, the key for es02 is in the results.
---
### Author:  [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [October 8, 2019, 4:23pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/9 "2019-10-08T16:23:36Z")
Okay, great!
Now let's make sure it's reporting at the proper time intervals:
```auto
POST .monitoring-es-*/_search
{
"size": 0,
"query": {
"bool": {
"filter": [
{
"term": {
"type": "node_stats"
}
},
{
"range": {
"timestamp": {
"gte": "now-1h"
}
}
}
]
}
},
"aggs": {
"nodes": {
"terms": {
"field": "node_stats.node_id",
"size": 100
},
"aggs": {
"timestamp": {
"terms": {
"field": "timestamp",
"size": 100
}
}
}
}
}
}
```
Let's make sure it's there again before moving to the next step. Please paste the entire output
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [October 8, 2019, 5:14pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/10 "2019-10-08T17:14:19Z")
Paste 4,104 lines?
---
### Author:  [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [October 8, 2019, 5:15pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/11 "2019-10-08T17:15:13Z")
Feel free to use a gist or something. I want to make sure we don't miss anything
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [October 8, 2019, 5:35pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/12 "2019-10-08T17:35:43Z")
I think.
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [October 16, 2019, 1:12pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/13 "2019-10-16T13:12:53Z")
Any additional thoughts?
---
### Author:  [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)
#### Post date: [October 21, 2019, 1:06pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/14 "2019-10-21T13:06:32Z")
@thealy I don't think we ever saw the gist get posted. Did you link it here?
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [October 21, 2019, 1:38pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/15 "2019-10-21T13:38:40Z")
Let's try this one: [https://gist.github.com/thealy/52bdad6fc825e4f392798b5e654851cd](https://gist.github.com/thealy/52bdad6fc825e4f392798b5e654851cd)
---
### Author:  [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)
#### Post date: [October 28, 2019, 10:24am UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/16 "2019-10-28T10:24:35Z")
Sorry for the delay here. In looking at this gist, it does appear that @chrisronline is on the right track. I see 8 of the 9 machines in this list and es02 is indeed missing. That suggests that although the data may be indexed, it may have an incorrect time stamp. Might it be the case that es02 is somehow an outlier with the timestamps it is setting?
---
### Author:  [@thealy](https://discuss.elastic.co/u/thealy)
#### Post date: [October 28, 2019, 1:31pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/17 "2019-10-28T13:31:38Z")
Spot on! Restarted NTP and up it came. Sorry for it to be something so lame, and appreciate the help in getting it back.
---
### Author:  [@Mike\_Place](https://discuss.elastic.co/u/Mike_Place)
#### Post date: [October 28, 2019, 2:04pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/18 "2019-10-28T14:04:49Z")
That's great news! Glad things are working for you now. Thanks, @thealy
---
### Author:  [@sharry007](https://discuss.elastic.co/u/sharry007)
#### Post date: [November 5, 2019, 4:16am UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/19 "2019-11-05T04:16:38Z")
@Mike_Place and @chrisronline,
We are also facing similar problem. Our cluster comprises of 3 master, 3 coordinate, 4 hot and 4 warm on 6.7.1 version. We upgrade one master and one warm node to 6.8.3 but those are not visible in Kibana monitoring console.
1. Output of
is
```
{
"aggregations" : {
"node_ids" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "XZFg8VpuRS2WIse9KBIvMA",
"doc_count" : 51900
},
{
"key" : "ebvjKCrhQvK8581Uvr8Gew",
"doc_count" : 51900
},
{
"key" : "xDtX5E1JRnu2BNv8zHe8bQ",
"doc_count" : 51900
},
{
"key" : "yvOSip5mQRG3uThwezXGBg",
"doc_count" : 51899
},
{
"key" : "GdXTW8qmR2aDXclXrQAgRw",
"doc_count" : 51897
},
{
"key" : "Qfpu2zrJTHaP_ngUNLZ7Jw",
"doc_count" : 51897
},
{
"key" : "A8Qe1_q3QUqYPsgw3jcz5g",
"doc_count" : 51896
},
{
"key" : "2OazDzlySLWXeTlNlkQ-cA",
"doc_count" : 51895
},
{
"key" : "XYIU3i3cRw-C7jAdDRj17g",
"doc_count" : 51895
},
{
"key" : "gdoeR2EzTvCuE4RDeeb_aA",
"doc_count" : 51895
},
{
"key" : "djCtwvYuSgWcgQ2DnbvitQ",
"doc_count" : 51893
},
{
"key" : "JNSXpU_1SP-nYn5lxWxZXQ",
"doc_count" : 51890
},
{
"key" : "a5_1IeuxSji6QYlo3SHQ6Q",
"doc_count" : 44169
},
{
"key" : "m_7bmIwbRoi-j80tDms2lw",
"doc_count" : 19446
}
]
}
}
}
```
1. Output of
is
>
please suggest..
---
### Author:  [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [November 5, 2019, 1:01pm UTC](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212/20 "2019-11-05T13:01:53Z")
> We upgrade one master and one warm node to 6.8.3
What are the node ids for these particular nodes? Do you see them in either output?
[Next page](https://discuss.elastic.co/t/one-es-node-not-showing-in-kibana-monitoring/200212.md?page=2)