# Direct correlation between pending tasks and GC

**URL:** <https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704>\
**Category:** Elasticsearch\
**Created:** [January 11, 2021, 10:42am UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704 "2021-01-11T10:42:43Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![tutunak](https://avatars.discourse-cdn.com/v4/letter/t/c67d28/32.png) [@tutunak](https://discuss.elastic.co/u/tutunak)\
**Post date:** [January 11, 2021, 10:42am UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/1 "2021-01-11T10:42:43Z")

</div>

Hi for all my clusters I see the direct correlation between pending tasks and GC time.

 ![3 hours pending tasks bluried](https://us1.discourse-cdn.com/elastic/original/3X/c/a/cac60cc4df06be0d15125bc7b18b05fe958d82f7.png)  
 ![24 hours pending tasks bluried](https://us1.discourse-cdn.com/elastic/original/3X/6/3/63510871e25fe6f6b3afa1f6f586ec48894fd8c1.png)  
What can I do with that? How can avoid this situation?

---

<div class="post-metadata">

**Author:** ![tutunak](https://avatars.discourse-cdn.com/v4/letter/t/c67d28/32.png) [@tutunak](https://discuss.elastic.co/u/tutunak)\
**Post date:** [January 12, 2021, 12:08pm UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/2 "2021-01-12T12:08:45Z")

</div>

Most of the pending tasks are snapshot related. We are decided to disable snapshots.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [January 12, 2021, 2:30pm UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/3 "2021-01-12T14:30:47Z")

</div>

Snapshots are the [last line of defence against data loss](https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability.html) so its [important to use them](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#important-settings-backups).

What's the actual issue here? A few ms of GC every 30 minutes or so doesn't sound like a problem that needs solving, and disabling snapshots will cause pain in future.

---

<div class="post-metadata">

**Author:** ![tutunak](https://avatars.discourse-cdn.com/v4/letter/t/c67d28/32.png) [@tutunak](https://discuss.elastic.co/u/tutunak)\
**Post date:** [January 16, 2021, 3:42pm UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/4 "2021-01-16T15:42:35Z")

</div>

Snapshots aren't mandatory for us. We can easily restore data. The main problem, that our application stop writes to elastic search when we have pending tasks. In this situation, we have to "freeze" work to 5 minutes every hour and it isn't acceptable.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [January 16, 2021, 4:17pm UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/5 "2021-01-16T16:17:00Z")

</div>

> [@tutunak](#):
>
> our application stop writes to Elasticsearch when we have pending tasks

Pending tasks are processed on the master and are therefore always a bottleneck since there's only ever one master. If you are performance-sensitive then you should definitely avoid having master-related actions on your indexing pathway - this mainly means dynamic index creation and dynamic mapping updates.

I also don't see how a few ms of GC translates into a five-minute "freeze", that's definitely not normal.

---

<div class="post-metadata">

**Author:** ![tutunak](https://avatars.discourse-cdn.com/v4/letter/t/c67d28/32.png) [@tutunak](https://discuss.elastic.co/u/tutunak)\
**Post date:** [January 17, 2021, 12:18pm UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/6 "2021-01-17T12:18:59Z")

</div>

There are minutes when GC works (spikes on the graphs). During this time you can see pending tasks spike. Our current solution stops write the data to the elastic if sees that we have pending tasks. From the elastic side, I suppose data will proceed. But we overloaded it in the past, so we decided to make the current solution as you see. I'm not sure that it was the right solution.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [January 17, 2021, 12:23pm UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/7 "2021-01-17T12:23:31Z")

</div>

> [@tutunak](#):
>
> Our current solution stops write the data to the elastic if sees that we have pending tasks.

Ok, that makes more sense. Don't do that. You absolutely can carry on indexing whether the master is busy or not.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 14, 2021, 12:23pm UTC](https://discuss.elastic.co/t/direct-correlation-between-pending-tasks-and-gc/260704/8 "2021-02-14T12:23:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
