# Stack Monitoring not showing data node statistics **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-monitoring\ **Created:** [February 16, 2020, 2:04am UTC](https://discuss.elastic.co/t/stack-monitoring-not-showing-data-node-statistics/219497 "2020-02-16T02:04:14Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)\ **Post date:** [February 16, 2020, 2:04am UTC](https://discuss.elastic.co/t/stack-monitoring-not-showing-data-node-statistics/219497/1 "2020-02-16T02:04:14Z")
For some reason, Stack Monitoring is now showing "Total shards: 0", "Documents: 0", "Data: 0.0B", and the node count is not including my data nodes. Why is no data present for any of my data nodes? If I restart Elasticsearch on one of the data nodes, it will briefly increase the node count by 1, but it then returns back to not including that node. I have seen a few suggestions to check the time on the servers, and they are all correct. Not sure where else to look to figure this out. Elasticsearch 7.6.0 Kibana 7.6.0 ```auto GET _cluster/health?pretty { "cluster_name" : "Cluster-Name", "status" : "green", "timed_out" : false, "number_of_nodes" : 9, "number_of_data_nodes" : 5, "active_primary_shards" : 360, "active_shards" : 720, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 } ``` ```auto GET _nodes/stats { "_nodes" : { "total" : 9, "successful" : 4, "failed" : 5, "failures" : [ { "type" : "failed_node_exception", "reason" : "Failed node [NzA6DLQoSvyYPZpC1_nc_w]", "node_id" : "NzA6DLQoSvyYPZpC1_nc_w", "caused_by" : { "type" : "illegal_state_exception", "reason" : "Negative longs unsupported, use writeLong or writeZLong for negative numbers [-410]" } }, { "type" : "failed_node_exception", "reason" : "Failed node [vzLH-Rc3RtqbFVnh6j39Cw]", "node_id" : "vzLH-Rc3RtqbFVnh6j39Cw", "caused_by" : { "type" : "illegal_state_exception", "reason" : "Negative longs unsupported, use writeLong or writeZLong for negative numbers [-2449]" } }, { "type" : "failed_node_exception", "reason" : "Failed node [mVi86ywZQ8e8cWJ2ype9cQ]", "node_id" : "mVi86ywZQ8e8cWJ2ype9cQ", "caused_by" : { "type" : "illegal_state_exception", "reason" : "Negative longs unsupported, use writeLong or writeZLong for negative numbers [-116]" } }, { "type" : "failed_node_exception", "reason" : "Failed node [B9I4onxkSRCHnQsl_VdX2Q]", "node_id" : "B9I4onxkSRCHnQsl_VdX2Q", "caused_by" : { "type" : "illegal_state_exception", "reason" : "Negative longs unsupported, use writeLong or writeZLong for negative numbers [-62]" } }, { "type" : "failed_node_exception", "reason" : "Failed node [rnj8pp48S1mX2fNej_TM-w]", "node_id" : "rnj8pp48S1mX2fNej_TM-w", "caused_by" : { "type" : "illegal_state_exception", "reason" : "Negative longs unsupported, use writeLong or writeZLong for negative numbers [-227]" } } ] }, "cluster_name" : "Cluster-Name", "nodes" : { THE FOUR UP NODES } } ``` When I look in Kibana, it shows 4 nodes (3x master and 1x coordinator), and no counts for shards, nodes, data, etc.. All the nodes do appear listed properly as up on the bottom of the page. ---
**Author:** ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)\ **Post date:** [February 16, 2020, 5:01am UTC](https://discuss.elastic.co/t/stack-monitoring-not-showing-data-node-statistics/219497/2 "2020-02-16T05:01:18Z")
Looks like this is the source of the error in the \_node/stats message. Found this in the elasticsearch.server logs. Issue with authentication? \ ```auto send message failed [channel: Netty4TcpChannel{localAddress=/:9300, remoteAddress=/:50402}] java.lang.IllegalStateException: Negative longs unsupported, use writeLong or writeZLong for negative numbers [-42] at org.elasticsearch.common.io.stream.StreamOutput.writeVLong(StreamOutput.java:299) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.ingest.IngestStats$Stats.writeTo(IngestStats.java:197) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.ingest.IngestStats.writeTo(IngestStats.java:103) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.common.io.stream.StreamOutput.writeOptionalWriteable(StreamOutput.java:897) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.action.admin.cluster.node.stats.NodeStats.writeTo(NodeStats.java:255) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundMessage.writeMessage(OutboundMessage.java:87) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundMessage.serialize(OutboundMessage.java:64) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundHandler$MessageSerializer.get(OutboundHandler.java:166) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundHandler$MessageSerializer.get(OutboundHandler.java:152) ~[elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundHandler$SendContext.get(OutboundHandler.java:199) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundHandler.internalSend(OutboundHandler.java:129) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundHandler.sendMessage(OutboundHandler.java:124) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.OutboundHandler.sendResponse(OutboundHandler.java:104) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.TcpTransportChannel.sendResponse(TcpTransportChannel.java:64) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:54) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:244) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:240) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:257) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.lambda$messageReceived$0(SecurityServerTransportInterceptor.java:306) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$4(AuthorizationService.java:241) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:616) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:591) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.RBACEngine.authorizeClusterAction(RBACEngine.java:144) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:243) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:227) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$1(AuthorizationService.java:193) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$1(RBACEngine.java:117) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:208) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.RBACEngine.getRoles(RBACEngine.java:123) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:111) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:195) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.lambda$inbound$1(ServerTransportFilter.java:130) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:248) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$6(AuthenticationService.java:310) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:321) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:245) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:196) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:139) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:121) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:313) [x-pack-security-7.6.0.jar:7.6.0] at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:63) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.transport.InboundHandler$RequestHandler.doRun(InboundHandler.java:264) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692) [elasticsearch-7.6.0.jar:7.6.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.0.jar:7.6.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:830) [?:?] ``` ---
**Author:** ![MakoWish](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@MakoWish](https://discuss.elastic.co/u/MakoWish)\ **Post date:** [February 16, 2020, 5:50am UTC](https://discuss.elastic.co/t/stack-monitoring-not-showing-data-node-statistics/219497/3 "2020-02-16T05:50:08Z")
Looks to possibly be related to [https://github.com/elastic/elasticsearch/issues/52339](https://github.com/elastic/elasticsearch/issues/52339) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 15, 2020, 5:50am UTC](https://discuss.elastic.co/t/stack-monitoring-not-showing-data-node-statistics/219497/4 "2020-03-15T05:50:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.