Unable to limit records using process.include_top_n

[safehouse]

I want to limit the records sent from metricbeat using the process.include_top_n set of options. However, wither I can't figure out the correct setup (likely) or it's not working as expected on my systems. Any help or guidance would be greatly appreciated.

here is part of the metricbeat yml file. Let me know if you would like to see more or different data:

#==========================  Modules configuration ============================
metricbeat.modules:
- module: system
  period: 30s
  metricsets:
    - cpu
    - load
    - memory
    - network
    - process
    #- process_summary
    #- core
    #- diskio
    #- socket
  processes: ['.*']
  process.include_top_n:
    by_cpu: 5      # include top 5 processes by CPU
    by_memory: 5   # include top 5 processes by memory

- module: system
  period: 5m
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

#- module: system
#  period: 15m
#  metricsets:
#    - uptime

  # If false, cmdline of a process is not cached.
  #process.cmdline.cache.enabled: true

  # Enable collection of cgroup metrics from processes on Linux.
  #process.cgroups.enabled: true

  # A list of regular expressions used to whitelist environment variables
  # reported with the process metricset's events. Defaults to empty.
  #process.env.whitelist: []

  # Include the cumulative CPU tick values with the process metrics. Defaults
  # to false.
  #process.include_cpu_ticks: false

  # Configure reverse DNS lookup on remote IP addresses in the socket metricset.
  #socket.reverse_lookup.enabled: false
  #socket.reverse_lookup.success_ttl: 60s
  #socket.reverse_lookup.failure_ttl: 60s

What I'm getting is all process data (or that's what it looks like to me)

I was expecting to see just 5 - 10 processes listed. Maybe these would grow over time as the top 5 changed and are recorded - but this data is from 15 minutes.

[steffens (Steffen Siering)]

Which version of metricbeat are you using?

[safehouse]

@steffens - good point. I forgot to mention I was testing with Metricbeat 5.6.2 and then also updated to 5.6.3-1 to test the latest version.

[andrewkroh]

process.include_top_n is a 6.0 feature. You'll see that it's only present in the 6.0 documentation. You can try the feature by downloading 6.0.0-rc1.

[safehouse]

Well, that would explain it! Thanks @andrewkroh

I'll update and report back here (hopefully with success).

[safehouse]

After updating to 6.0.0-rc1 it's working as expected - and that is awesome. Saving me tons of disk space and helping me focus on just the data I need. Thanks!