# Filebeat putting too much load on the system. Seems to be related to gc.log

**URL:** <https://discuss.elastic.co/t/filebeat-putting-too-much-load-on-the-system-seems-to-be-related-to-gc-log/238966>\
**Category:** Beats\
**Created:** [June 27, 2020, 5:58pm UTC](https://discuss.elastic.co/t/filebeat-putting-too-much-load-on-the-system-seems-to-be-related-to-gc-log/238966 "2020-06-27T17:58:08Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [June 27, 2020, 5:58pm UTC](https://discuss.elastic.co/t/filebeat-putting-too-much-load-on-the-system-seems-to-be-related-to-gc-log/238966/1 "2020-06-27T17:58:08Z")

</div>

On our elastic 7.7.1 stack, when we enabled our elastic log ingestion via filebeat, we saw a significant load on our data and ingestion nodes.

Our log path parameter was defined as /var/log/elasticsearch/\*.json and all other parameters were at their respective default values.

After much trial and error, we determined that it was most likely being caused by the presence of old garbage collector log files which were in the format gc.log.nn (gc.log.01, gc.log.02 etc)

We tried to prevent filebeat from reading these files by adding regex to exclude these files

> exclude\_files: ['.gz$', '^gc']

but that did not have any impact. When we deleted those gc logs, the problem disappeared.

So I am wondering why is filebeat parsing those gc log files (assuming our assessment is true) when there is no directive in yaml config to do so.

---

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [June 28, 2020, 1:28am UTC](https://discuss.elastic.co/t/filebeat-putting-too-much-load-on-the-system-seems-to-be-related-to-gc-log/238966/2 "2020-06-28T01:28:54Z")

</div>

I forgot to add this to my earlier post.

When I look at the filebeat index, it already has 48 million documents in it and almost all of them are from ingesting gc.log files from various nodes.

So, my original question remains: How do I prevent filebeat from ingesting gc.log ?

---

<div class="post-metadata">

**Author:** ![Webtrend\_Inc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/webtrend_inc/32/70034_2.png) [@Webtrend\_Inc](https://discuss.elastic.co/u/Webtrend_Inc)\
**Post date:** [June 29, 2020, 7:46am UTC](https://discuss.elastic.co/t/filebeat-putting-too-much-load-on-the-system-seems-to-be-related-to-gc-log/238966/3 "2020-06-29T07:46:20Z")

</div>

Answering my own question: I had the elasticsearch module enabled and that is why it was parsing gc logs

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 27, 2020, 9:46am UTC](https://discuss.elastic.co/t/filebeat-putting-too-much-load-on-the-system-seems-to-be-related-to-gc-log/238966/4 "2020-07-27T09:46:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.