I'm trying to give Filebeat as little memory as possible. My main issue is that I have a ton of independent containers on a VM and sometimes (due to network issues) Filebeat can't forward all the logs. However, after the issue is solved all (hundreds) Filebeat instances are throwing data to memory like crazy and that brings down the VM itself. I tried to use disk queue, but it didn't have any effect on the memory usage. I'm monitoring the memory usage with Metricbeat.
Do you have any suggestions limiting Filebeat's memory usage? Or what am I missing when I add the disk queue to the configuration?
Filebeat version: 7.11.0