Controlling memory and cpu footprint of filebeat

Elastic Stack Beats
1

Can we somehow control

  1. No of threads used by filebeat.
  2. CPU utilization, other than through scan_frequency.
  3. Memory utilization.
2

This is currently not possible. Perhaps you can give some more insights in what your problem is to potentially find some other solutions?

3

I am planning to use filebeat in production and want to see the cpu and memory footprint of filebeat, Is there any tool for monitoring Filebeat resource utilizations. And are there any settings through which i can alter the cpu and memory utilizations of Filebeat.

4

There are 'some' settings, but in general it's recommended to use OS tools to limit filebeat resource usage.

e.g. environment variable GOMAXPROCS=... sets number of active worker threads in golang runtime environment. Using load-balancing and publish_async: true potentially increases memory usage if log files are written fast enough.

for linux check out:

  • nice
  • ionice
  • taskset
  • systemd resource control
  • cgroups

When limiting resource usage, make sure filebeat still can catch up at less busy times.

You can use topbeat to monitor topbeat/filebeat resource usage :wink:

1 Like
5

Oh! this helps a lot, thanks steffens