# Filebeat High CPU consumption on two of four hosts

**URL:** https://discuss.elastic.co/t/filebeat-high-cpu-consumption-on-two-of-four-hosts/183399
**Category:** Beats
**Tags:** filebeat
**Created:** [May 29, 2019, 5:52pm UTC](https://discuss.elastic.co/t/filebeat-high-cpu-consumption-on-two-of-four-hosts/183399 "2019-05-29T17:52:03Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Aross](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@Aross](https://discuss.elastic.co/u/Aross)
#### Post date: [May 29, 2019, 5:52pm UTC](https://discuss.elastic.co/t/filebeat-high-cpu-consumption-on-two-of-four-hosts/183399/1 "2019-05-29T17:52:03Z")

</div>

Hi,

I am using filebeat version 6.2.4 on a RHEL host to ship log data from applications to a log stash instance. On two of our four hosts that are configured to send data to the same log stash hosts, we are seeing significantly higher CPU usage (less than 30% compared to 150%+). The configuration files between these hosts are identical, however logs being captured by the glob patterns could vary in size and content due to different applications running between the hosts.

Looking through previous posts regarding high CPU, I have changed the scan\_frequency to 30s up from 10s, and set max\_procs: 2 with only mild improvements. CPU was above 200% prior to the max\_procs setting. Our registry file is about 2.9M in size, which may seem large but the hosts using less than 30% CPU on average have a similar registry file size.

When comparing the monitoring output in logs, the number of open files and volume of data appear to be similar between hosts, ranging from 90 to 120 open files, so the difference in performance has me confused as to what else could be contributing to the significant difference in resources. Would you have any suggestions on what to look at next?

---

<div class="post-metadata">

### Author: ![Aross](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@Aross](https://discuss.elastic.co/u/Aross)
#### Post date: [June 3, 2019, 2:22pm UTC](https://discuss.elastic.co/t/filebeat-high-cpu-consumption-on-two-of-four-hosts/183399/2 "2019-06-03T14:22:36Z")

</div>

For anyone else that stumbles upon this post...

There were some extremely large log files in our environment due to trace and debug logging enabled for logs that filebeat was capturing. I think we will be able to narrow in on all log files that send a significant amount of data using the debug logging filebeat has for publish events.

---

<div class="post-metadata">

### Author: ![vasek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vasek/32/136636_2.png) [@vasek](https://discuss.elastic.co/u/vasek)
#### Post date: [June 17, 2019, 7:48am UTC](https://discuss.elastic.co/t/filebeat-high-cpu-consumption-on-two-of-four-hosts/183399/3 "2019-06-17T07:48:36Z")

</div>

I have some solutions for you:

**1) Download log files to non-production environment**  
By non-production environment I mean a server that does not negatively affect the service.

Use FB (filebeat) for sending these big files to your Log Management. You can use this [tutorial](https://discuss.elastic.co/t/filebeat-on-demand-loading-compressed-gz-files-from-stdin/184159).

**2) CPU load by filebeat with respect to running application**  
You can set lowest priority of consuming CPU for Filebeat service. [Lowest priority is 19](https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_MRG/1.2/html/Realtime_Reference_Guide/chap-Realtime_Reference_Guide-Priorities_and_policies.html).

```
vim /usr/lib/systemd/system/filebeat.service
[Service]
Nice=19

```

This is very useful in production environment.

Hope it will be useful for you.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 15, 2019, 7:48am UTC](https://discuss.elastic.co/t/filebeat-high-cpu-consumption-on-two-of-four-hosts/183399/4 "2019-07-15T07:48:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
