# Failed to retrieve shards stuck Elastic node and performance

**URL:** <https://discuss.elastic.co/t/failed-to-retrieve-shards-stuck-elastic-node-and-performance/361658>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-monitoring\
**Created:** [June 18, 2024, 2:52pm UTC](https://discuss.elastic.co/t/failed-to-retrieve-shards-stuck-elastic-node-and-performance/361658 "2024-06-18T14:52:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![todorvgit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/todorvgit/32/135370_2.png) [@todorvgit](https://discuss.elastic.co/u/todorvgit)\
**Post date:** [June 18, 2024, 2:52pm UTC](https://discuss.elastic.co/t/failed-to-retrieve-shards-stuck-elastic-node-and-performance/361658/1 "2024-06-18T14:52:38Z")

</div>

Hello,

We are having issue with Hot tier in our elasticsearch version 7.16.2  
Time to time in last few weeks we are experience issue that Metrics beat stop getting any data of the cluster.  
What we are seeing in master is that some of nodes failed with:  
Failed to retrieve shards from node.  
"type": "server", "timestamp": "2024-06-18T14:27:34,223Z", "level": "WARN", "component": "o.e.c.InternalClusterInfoService", "cluster.name": "eck-elasticsearch", "node.name": "eck-elasticsearch-es-master-2", "message": "failed to retrieve stats for node [nodename]: [eck-elasticsearch-es-hot-v8-15][1.1.1.2:9300][cluster:monitor/nodes/stats[n]] request\_id [272221826] timed out after [15007ms]", "cluster.uuid": "sdasdasd", "node.id": "sddsaddvdNOTA2Kin37RdKnxw" }

When we restart a pod issue get fixed temproary.

What we found is time to time our nodes hit max Peak Throughput 1200 MB/s combined for Read/Write

We are using pd-ssd tier on Google cloud n1 tier which one of the highest.

We also have been set on our indices `index.refresh_interval` to be 30 sec.