Metricbeat 8.11.0 - system module using excessive amount of memory

[twilson]

I'm using the system integration with the Elastic agent and have the ' Collect metrics from System instances' option enabled, with all the default datasets within that selected. With the 8.11.0 version of this integration the amount of memory the metricbeat process consumes continues to grow over time, in to the GB range.

I've narrowed this down to two specific metric sets - 'System process metrics' and 'System process_summary metrics'. When these two metric sets are disabled the memory usage remains low and constant. The following information was pulled using 'tasklist /v'.

Memory usage after 12 hours with the two suspect metric sets disabled:

Image Name                     PID Session Name        Session#    Mem Usage 
========================= ======== ================ =========== ============ 
metricbeat.exe               10468 Services                   0    122,036 K

Memory usage after 1 hour with the two suspect metric sets enabled:

Image Name                     PID Session Name        Session#    Mem Usage 
========================= ======== ================ =========== ============ 
metricbeat.exe               10468 Services                   0  1,011,256 K

This is happening on all Windows machines with the 8.11.0 agent/metricbeat. During the time it took me to type this the memory usage has increased to 1,117,656 K. I've performed the tests of disabling/enabling the metrics multiple times and have reached the same results.

This is not happening with Windows machines running 8.10.4 agent/metricbeat. I've looked through the agent logs and could no nothing to indicate a problem.

[catn0b0t]

Jup, we just noticed the same thing. I pulled a graph of the average Metricbeat memory usage over all our hosts and this is what that looks like. The memory consumption starts at a normaal 0.5Gb and then grows to almost 4Gb overnight.

[catn0b0t]

Additional observation. I have been playing with the "System process metrics" period and this seems to have a big impact on the speed the memory leak grows. If i lower the period to 1s, I can see the memory usage by metrics grow by the second, it just doesn't releas any of it. When I raise the period to 30sec, I still see the memory only growing, but at a much slower pace.

[pikaia]

We are seeing the same behavior... Could this be a bug?

[Frank_Barton]

There is a confirmed bug

[metricbeat] memory leak 10GB RAM usage and crash, urgent solution needed