Unusually high Metricbeat memory usage

I have metricbeat installed on a Windows Server 2016 Datacenter server that also has an Elasticsearch node.

I also have metricbeat installed on a Windows Server 2012 Standard server with an Elasticsearch node as well in the same Elasticsearch cluster as the 2016 server.

Both on Elastic Stack 7.1.1.

Only the system module is enabled and configured identically on both servers:

# Module: system
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.1/metricbeat-module-system.html

- module: system
  period: 10s
  metricsets:
    - cpu
    #- load
    - memory
    - network
    - process
    - process_summary
    - socket_summary
    #- core
    #- diskio
    #- socket
  process.include_top_n:
    by_cpu: 20      # include top 20 processes by CPU
    by_memory: 20   # include top 20 processes by memory

- module: system
  period: 1m
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

- module: system
  period: 15m
  metricsets:
    - uptime

#- module: system
#  period: 5m
#  metricsets:
#    - raid
#  raid.mount_point: '/'

At this point metricbeat has been running for around 1 week, and the memory usage on the 2016 server is > 3GB:
image

On the 2012 server it is significantly lower:
image

Any clue as to why this is happening only on the 2016 server?
Area chart of the memory usage in the past week:

2 Likes

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

I too have the same issue with every 7.x Metricbeat I've tried. The only thing I've changed it the elasticsearch host from the default .yaml files. I did try changing the check period from 10s to 60s and that causes a much slower increase in RAM usage over time.

Capture

I increased the period for the first metricset from 10s to 60s and it does appear to slow down the growth:
Top graph is the 2016 server, bottom graph is the 2012 server.

I have also observed this, Metricbeat 7.1.1 running on Windows Server 2016 Data Center ramps up to > 3GB memory utilization.