# Stuck "Cancelled Tasks" In ElasticSearch 8.6.2 causing Cluster failure

**URL:** <https://discuss.elastic.co/t/stuck-cancelled-tasks-in-elasticsearch-8-6-2-causing-cluster-failure/337490>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [July 3, 2023, 5:26pm UTC](https://discuss.elastic.co/t/stuck-cancelled-tasks-in-elasticsearch-8-6-2-causing-cluster-failure/337490 "2023-07-03T17:26:15Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [July 3, 2023, 6:47pm UTC](https://discuss.elastic.co/t/stuck-cancelled-tasks-in-elasticsearch-8-6-2-causing-cluster-failure/337490/3 "2023-07-03T18:47:57Z")

</div>

> [@Thomas\_Kuisel](#):
>
> We think there are transient/sporadic connection issues occuring that are causing connections to be timed out between nodes and then the task is stuck waiting on a response indefinitely.

Hmm actually having said that I don't think it's a config issue, Linux has a silly default that means that it can take 900+ seconds between a connection drop and a notification to userspace about the connection drop. [This page of the manual](https://www.elastic.co/guide/en/elasticsearch/reference/current/system-config-tcpretries.html) has more details. You'd see messages about dropped connections in the logs if it was this.

You'd need to be a bit more precise about what you mean by "kill our cluster" tho. The [other recent thread](https://discuss.elastic.co/t/cancellation-of-tasks/337454/4) on this topic has the zombie tasks consuming a lot of CPU, but that wouldn't happen if they were just waiting for a dead connection to time out. They'd potentially hold on to a lot of heap, causing GC pressure, but wouldn't themselves consume any CPU.

---

_[View the full topic](https://discuss.elastic.co/t/stuck-cancelled-tasks-in-elasticsearch-8-6-2-causing-cluster-failure/337490)._
