Does anyone have a good method for monitoring that specific processes are running, even if said processes are not in the top N cpu/ram usage group?
The System integration lets you monitor the Top N processes. You can limit what processes are selected via a regex in the Processes inputs. But so far my attempts at manipulating that config into doing what I want haven't worked very well.
Specifically I added a second System integration and limited it to just .*(ssh|postfix|filebeat|metricbeat|auditbeat|dockerd|auditd).* processes. (I disabled all other collections.) Then I had to increase the number of top N processes to 1000 in order to make sure all the processes were grabbed.
In my normal system integration I tried to tell it to ignore any of the processes monitored by the other integration, but none of the regexes I tried seemed to do the trick. Specifically they seem to block all processes, not just the ones listed.
@stephenb So that does work. I'm always getting the processes I want. Thanks.
I also set up an ingest pipeline to drop duplicated events from the Top N configuration.
For anyone else who comes across this, you might see more results than you expect if your system has more than one instance of something running.
In my case i was seeing more records for each of the beats I have running than I expected. After looking into it, I figured out that there is an instance of each beat per configured Agent integration that uses the beat. So, instead of 6 per minute, I was seeing 24 per minute for metricbeat. ps shows 4 instances of metricbeat on my vm. So that works out.