# Filebeat creating write disk i/o when filtering

**URL:** <https://discuss.elastic.co/t/filebeat-creating-write-disk-i-o-when-filtering/342540>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 7, 2023, 4:57pm UTC](https://discuss.elastic.co/t/filebeat-creating-write-disk-i-o-when-filtering/342540 "2023-09-07T16:57:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rdang](https://avatars.discourse-cdn.com/v4/letter/r/c37758/32.png) [@rdang](https://discuss.elastic.co/u/rdang)\
**Post date:** [September 7, 2023, 4:57pm UTC](https://discuss.elastic.co/t/filebeat-creating-write-disk-i-o-when-filtering/342540/1 "2023-09-07T16:57:33Z")

</div>

Hi folks, we are using filebeat 6.8 on Ubuntu 18.04.5 LTS with ESM.  
Filebeat is reading from `mysql-audit.log` thats configured to log CONNECT and QUERY events. Filebeat sends to a load balancer fronting logstash receivers.

In our desire to reduce the amount of data filebeat sends to logstash without altering what mariadb logs to disk, we are testing the use of `include_lines` in the definition for that particular log file in filebeats configuration. It appears to be working as we are now seeing only the events with the matching regex set in filebeat config.

There were concerns expressed that adding the filter may increase CPU usage due to the sheer volume of data being processed from `mysql-audit.log`. This doesn't appear to be the case, however we did see a tripling of disk write I/O during the period of testing:

![Screen Shot 2023-09-07 at 12.55.01 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/7/37d48ba38e0e61c949b7f1483b4eaf0654b524a7.png)

Does filebeat cache to disk log data its read before applying filters?

---

<div class="post-metadata">

**Author:** ![rdang](https://avatars.discourse-cdn.com/v4/letter/r/c37758/32.png) [@rdang](https://discuss.elastic.co/u/rdang)\
**Post date:** [September 11, 2023, 1:43pm UTC](https://discuss.elastic.co/t/filebeat-creating-write-disk-i-o-when-filtering/342540/2 "2023-09-11T13:43:32Z")

</div>

We have determined so far that the disk write I/O is occurring on the root disk from which filebeat is configured and executed. When we stop filebeat, disk write I/O returns to level prior to starting it. Confirmed which disk using `iostat`. `strace` doesn't show any write activity.

Any ideas?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2023, 3:44pm UTC](https://discuss.elastic.co/t/filebeat-creating-write-disk-i-o-when-filtering/342540/3 "2023-10-09T15:44:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.