# Transient Network Outage and Cluster Health

**URL:** <https://discuss.elastic.co/t/transient-network-outage-and-cluster-health/3525>\
**Category:** Elasticsearch\
**Created:** [November 5, 2010, 2:00pm UTC](https://discuss.elastic.co/t/transient-network-outage-and-cluster-health/3525 "2010-11-05T14:00:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [November 5, 2010, 2:00pm UTC](https://discuss.elastic.co/t/transient-network-outage-and-cluster-health/3525/1 "2010-11-05T14:00:11Z")

</div>

Hi,

We have two clusters that both go to yellow when there is a transient  
network outage. This happens overnight mostly, perhaps some form of  
maintenance on the cloud providers part. The cluster never recovers the  
connection and both require a restart of their secondary nodes. Is there a  
setting I need to change in order to keep this from happening? The relevant  
part of the config file is:

cloud:

> ```
> aws:
> access_key: munged
> secret_key: munged
> 
> ```
> 
> gateway:  
> type: s3  
> s3:  
> bucket: munged  
> recover\_after\_nodes: 2
> 
> network:  
> host: host0
> 
> discovery.zen.ping.multicast:  
> enabled: false
> 
> discovery.zen.ping.unicast:  
> hosts: ["host0:9300","host1:9300"]

...Thanks,  
...Ken

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [November 5, 2010, 4:38pm UTC](https://discuss.elastic.co/t/transient-network-outage-and-cluster-health/3525/2 "2010-11-05T16:38:09Z")

</div>

Currently, when a node gets disconnected from a cluster, it requires a  
restart in order to rejoin the cluster, it does not join the cluster  
automatically. I am working on improving on that... .

For now, maybe just increase the default fault detection timeouts? Check  
this:  
[http://www.elasticsearch.com/docs/elasticsearch/modules/discovery/zen/#Fault\_Detection](http://www.elasticsearch.com/docs/elasticsearch/modules/discovery/zen/#Fault_Detection).  
What is the message that you get in the log when it gets disconnected?

-shay.banon

On Fri, Nov 5, 2010 at 4:00 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com) wrote:

> Hi,
> 
> We have two clusters that both go to yellow when there is a transient  
> network outage. This happens overnight mostly, perhaps some form of  
> maintenance on the cloud providers part. The cluster never recovers the  
> connection and both require a restart of their secondary nodes. Is there a  
> setting I need to change in order to keep this from happening? The relevant  
> part of the config file is:
> 
> cloud:
> 
> > ```
> > aws:
> > access_key: munged
> > secret_key: munged
> > 
> > ```
> > 
> > gateway:  
> > type: s3  
> > s3:  
> > bucket: munged  
> > recover\_after\_nodes: 2
> > 
> > network:  
> > host: host0
> > 
> > discovery.zen.ping.multicast:  
> > enabled: false
> > 
> > discovery.zen.ping.unicast:  
> > hosts: ["host0:9300","host1:9300"]
> 
> ...Thanks,  
> ...Ken

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:17am UTC](https://discuss.elastic.co/t/transient-network-outage-and-cluster-health/3525/3 "2017-07-06T04:17:03Z")

</div>