# Message "timeout notification from cluster service"

**URL:** <https://discuss.elastic.co/t/message-timeout-notification-from-cluster-service/79383>\
**Category:** Elasticsearch\
**Created:** [March 21, 2017, 10:38am UTC](https://discuss.elastic.co/t/message-timeout-notification-from-cluster-service/79383 "2017-03-21T10:38:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rghv](https://avatars.discourse-cdn.com/v4/letter/r/a87d85/32.png) [@rghv](https://discuss.elastic.co/u/rghv)\
**Post date:** [March 21, 2017, 10:38am UTC](https://discuss.elastic.co/t/message-timeout-notification-from-cluster-service/79383/1 "2017-03-21T10:38:51Z")

</div>

One of our nodes (N1) has been very frequently reporting the message "timeout notification from cluster service". I referred to [ElasticSearch : observer: timeout notification from cluster service](https://discuss.elastic.co/t/elasticsearch-observer-timeout-notification-from-cluster-service/32904) to check if that's got anything to do with gc on the other node (N2) in the cluster. The other node in the cluster doesn't have any slow gc runs, only one such gc run is logged. Where as N1 logs report about 20K such timeout messages over a windows of 24 hours.

Other than the gc run in progress, what else could a node(N1 in this case) to throw "timeout notification from cluster service" while waiting for the other node(N2)? Would it happen if the load on the other node (N2) was too high?

Please share your thoughts!

---

<div class="post-metadata">

**Author:** ![rghv](https://avatars.discourse-cdn.com/v4/letter/r/a87d85/32.png) [@rghv](https://discuss.elastic.co/u/rghv)\
**Post date:** [March 21, 2017, 10:59am UTC](https://discuss.elastic.co/t/message-timeout-notification-from-cluster-service/79383/2 "2017-03-21T10:59:43Z")

</div>

As seen on logs of N1

```
es@node1:/var/log/elasticsearch$ grep -irs "timeout notification from cluster service" es-n1.log |egrep "2017-03-21 [0-2][0-9]:"|wc -l
18479
es@node1:/var/log/elasticsearch$ grep -irs "timeout notification from cluster service" es-n1.log |egrep "2017-03-21 1[0-9]:"|wc -l
4
es@node1:/var/log/elasticsearch$ 

```

As seen on logs of N2

```
es@node2:/var/log/elasticsearch$ egrep "1[0-9]:[0-9][0-9]:" es-n2.log | egrep -iw gc
es@node2:/var/log/elasticsearch$ 

```

About 18K such messages seen on N1 where as logs in N2 do not hint at any trouble.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 21, 2017, 9:37pm UTC](https://discuss.elastic.co/t/message-timeout-notification-from-cluster-service/79383/3 "2017-03-21T21:37:21Z")

</div>

Are they in the same datacentre/region?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 18, 2017, 9:37pm UTC](https://discuss.elastic.co/t/message-timeout-notification-from-cluster-service/79383/4 "2017-04-18T21:37:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.