Iterating over the same grok filter

Elastic Stack Logstash
1

I am a new user here, and don't know if this is a trivial question or not (it certainly is not for me).

What I want to do is create a filter that creates fields with names based on the data itself. I think I can do this easily enough, but I need to recursively iterate over this, which is what is tripping me up. I want to take input like this:
'process_name="ARP" cpu=12.53 process_name="SNMP" cpu=8.37'
and produce the fields: ARP_cpu=12.53 and SNMP_cpu=8.37

(important note: I do not know what these process names are ahead of time, and all of the processes are listed on the same line).
This is my current logstash.conf:

filter {

  grok {
    match => ["message", "process_name=\"(?<ProcName>.*?)\" cpu=%{NUMBER:procCpu:float}"]
  }
  if ([ProcName]) {
    mutate{
      add_field => {
        "%{ProcName}_cpu" => "%{procCpu}"
      }
    }
  }
}

which works fine for the first process, but I need to run the input thru this filter multiple times until there are no more matches.
Is this easily done, or does anyone have better suggestions on how to do this?

Thanks

2

If your input has this format:

process_name="ARP" cpu=12.53 process_name="SNMP" cpu=8.37

The best way to parse it is using the kv filter, as this message is composed of key-value pairs.

Try the following kv filter instead of the grok filter.

kv {
    source => "message"
}
3

grok only matches once, you will have to use String .scan in a ruby filter...

    ruby {
        code => '
            msg = event.get("message")
            if msg
                matches = msg.scan(/process_name="([^"]+)" cpu=([0-9\.]+(\s|$))/)
                matches.each { |x|
                    event.set("#{x[0]}_cpu", x[1].to_f)
                }
            end
        '
    }

to get

  "SNMP_cpu" => 8.37,
   "ARP_cpu" => 12.53
4

Thanks for the help!

5

Thanks, that did the trick!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.