# Cluster has become unresponsive **URL:** \ **Category:** Elasticsearch\ **Created:** [January 23, 2019, 9:04pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485 "2019-01-23T21:04:26Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![connor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/connor/32/36606_2.png) [@connor](https://discuss.elastic.co/u/connor)\ **Post date:** [January 23, 2019, 9:04pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/1 "2019-01-23T21:04:26Z")
Our ES service has all of a sudden become unresponsive. HEAD is unable to connect to it, I am able to execute cluster health against it, but any stats just remain spinning, ES does not return the results. Is there anyone that can help diagnose the problem? ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [January 23, 2019, 9:45pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/2 "2019-01-23T21:45:23Z")
are you still on 2.3.1 or 1.3.2 or newer? ---
**Author:** ![connor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/connor/32/36606_2.png) [@connor](https://discuss.elastic.co/u/connor)\ **Post date:** [January 23, 2019, 9:46pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/3 "2019-01-23T21:46:23Z")
only seeing timeouts, no root cause indication. Is this group only for those versions? Running on 6.4.1 ---
**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\ **Post date:** [January 23, 2019, 9:51pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/4 "2019-01-23T21:51:15Z")
share some logs, otherwise it's hard to troubleshoot without seeing anything... ---
**Author:** ![connor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/connor/32/36606_2.png) [@connor](https://discuss.elastic.co/u/connor)\ **Post date:** [January 23, 2019, 9:54pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/5 "2019-01-23T21:54:28Z")
This is what all the timeouts look like [2019-01-23T16:55:38,324][WARN][o.e.t.TransportService] [uOeCjC6] Received response for a request that has timed out, sent [28687ms] ago, timed out [13687ms] ago, action [cluster:monitor/nodes/stats[n]], node [{9bcbJqh}{9bcbJqhRSbGJKbarmYDaYA}{b8lD9IQYQ\_CrVkOaLD7BBA}{10.0.2.89}{10.0.2.89:9300}{aws\_availability\_zone=eu-west-1c, ml.machine\_memory=32620150784, ml.max\_open\_jobs=20, xpack.installed=true, ml.enabled=true}], id [41894023] ---
**Author:** ![connor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/connor/32/36606_2.png) [@connor](https://discuss.elastic.co/u/connor)\ **Post date:** [January 23, 2019, 11:14pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/6 "2019-01-23T23:14:12Z")
I think it might have been caused by a high cardinality query returning a very large amount of unique items. Regardless, if I query cluster health it returns green, but if I try and execute any node stats or other node ops, it just doesn't return. How do I get the cluster back to normal? PS cluster is docker deployment on AWS ECS ---
**Author:** ![connor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/connor/32/36606_2.png) [@connor](https://discuss.elastic.co/u/connor)\ **Post date:** [January 24, 2019, 1:12am UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/7 "2019-01-24T01:12:42Z")
Adding more nodes to the data cluster brought all ES api's to life ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [January 24, 2019, 3:29pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/8 "2019-01-24T15:29:01Z")
> [@connor](#): > > I query cluster health it returns green, but if I try and execute any node stats or other node ops, it just doesn't return. Glad to see you got your cluster to be healthy again. In case you're interested, a cluster health request only normally goes to the master node, whereas things like node stats requests will fan out to every node in the cluster. If any node is too busy to respond then this kind of request can hang until that node recovers. A good thing to try is `GET /_nodes/hot_threads` since this gets a thread dump of every node in the cluster and will show you what the busy ones are busy doing, at a very low level. If you get into this state again then try that and share the output here. ---
**Author:** ![connor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/connor/32/36606_2.png) [@connor](https://discuss.elastic.co/u/connor)\ **Post date:** [January 24, 2019, 5:11pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/9 "2019-01-24T17:11:47Z")
Thanks David, will definitely add to my toolbox scripts! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 21, 2019, 5:11pm UTC](https://discuss.elastic.co/t/cluster-has-become-unresponsive/165485/10 "2019-02-21T17:11:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.