# Elasticsearch cluster crashed when 1 node got IO issues

**URL:** <https://discuss.elastic.co/t/elasticsearch-cluster-crashed-when-1-node-got-io-issues/208289>\
**Category:** Elasticsearch\
**Created:** [November 18, 2019, 10:52am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-crashed-when-1-node-got-io-issues/208289 "2019-11-18T10:52:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andriy\_Pyshchyk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andriy_pyshchyk/32/44334_2.png) [@Andriy\_Pyshchyk](https://discuss.elastic.co/u/Andriy_Pyshchyk)\
**Post date:** [November 18, 2019, 10:52am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-crashed-when-1-node-got-io-issues/208289/1 "2019-11-18T10:52:00Z")

</div>

Hi Everyone,

currently I'm using ES 6.6.1, in 48 nodes cluster, replication factor 1, and faced an issue when 1 of nodes had IO issues - whole cluster got to Red state, all indices got red, couldn't execute \_cat/nodes. In logs got lots of errors: [2019-11-16T19:50:59,398][WARN][r.suppressed] [es1-master-01-...] path: /.kibana/doc/kql-telemetry%3Akql-telemetry, params: {index=.kibana, id=kql-telemetry:kql-telemetry, type=doc}

And errors were not connected to node which actually failed

Only removing sick node helped cluster to recover. Is it any way to tell Elasticsearch to remove more from cluster if it doesn't respond for a while?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 16, 2019, 10:52am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-crashed-when-1-node-got-io-issues/208289/2 "2019-12-16T10:52:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.