- module: system
metricsets:
- process
enabled: true
period: 60s
processes: ['.*']
process.cgroups.enabled: false
processors:
- drop_event.when.range:
system.process.cpu.total.pct.lte: 1
I would like only the processes with a higher cpu than 1%. However it keeps pushing all the processes. What is the best way to do this in version 5.6.2?
To bad, it still gives me back 147 processes, all of them are around 0.01%.
Any idea's? Can it be because of processes: ['.*'] entry. I tried removing it, but then I get no output whatsoever.
I am using version 5.6.2 but both both does not seem to trigger.
My full list is this one.
#========================== Modules configuration ============================
metricbeat.modules:
#------------------------------- System Module -------------------------------
- module: system
metricsets:
- cpu
- core
- memory
- network
enabled: true
period: 300s
- module: system
metricsets:
- filesystem
- fsstat
- diskio
enabled: true
period: 10m
- module: system
metricsets:
- load
enabled: true
period: 60s
- module: system
metricsets:
- process
enabled: true
period: 60s
processes: ['.*']
process.cgroups.enabled: false
processors:
- drop_event:
when:
range:
system.process.cpu.total.pct.lte: 1
#================================ General =====================================
# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:
#----------------------------- Logstash output --------------------------------
output.logstash:
This is the output when I run with the parameter -e.
INFO Non-zero metrics in the last 30s: fetches.system-core.events=2 fetches.system-core.success=1 fetches.system-cpu.events=1 fetches.system-cpu.success=1 fetches.system-diskio.events=2 fetches.system-diskio.success=1 fetches.system-filesystem.events=31 fetches.system-filesystem.success=1 fetches.system-fsstat.events=1 fetches.system-fsstat.success=1 fetches.system-load.events=1 fetches.system-load.success=1 fetches.system-memory.events=1 fetches.system-memory.success=1 fetches.system-network.events=2 fetches.system-network.success=1 fetches.system-process.events=155 fetches.system-process.success=1 libbeat.logstash.call_count.PublishEvents=1 libbeat.logstash.publish.read_bytes=5137 libbeat.logstash.publish.write_bytes=15700 libbeat.logstash.published_and_acked_events=196 libbeat.publisher.messages_in_worker_queues=196 libbeat.publisher.published_events=196
I can see that it fetches 155 processes but no processors (filter) is trigger so it seems.
I put a '#' before processes: ['.*'], and keeps pushing all the processes. With monitoring several hundreds of servers it will take our disk space very fast.
The processors configuration the way you use it has been added to 6.0 release only. In 5.6 it works a little different. You have to use:
- module: system
...
filters:
- drop_event.when.range:
cpu.total.pct.lte: 1
The problem with 5.6 is, filtering happens on the not yet fully constructed event. That is, I'm not sure if you have to use system.process.cpu.total...., or just process.cpu.total... or just cpu.total..., or total...
I will try the filter. I got my code from the documentation from 5.6 as I can remember.
I see. This can very well be the case. But the processors setting in 5.6 is applied globally only, not per module. You have to remove all indentation to have something like:
No problem. In the end this was the solution for me:
- module: system
metricsets:
- process
enabled: true
period: 15s
process.cgroups.enabled: false
filters:
- drop_event.when.range:
cpu.total.pct.lte: 0.10
- module: system
metricsets:
- process
enabled: true
period: 15s
process.cgroups.enabled: false
filters:
- drop_event.when.range:
memory.rss.pct.lte: 0.10
I found out though that 0.10 stands for 10%, 1 is 100%. So 1% would be 0.01. So now all my system events per 15s are being being processed when memory is 10% or cpu is 10% or more. Maybe I can chain it, but for now this would suffice.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.