GROK multiple MATCH filters

Elastic Stack Logstash
1

Apologies if this has been asked repetitively, however I have a hard time understanding multiple MATCH filters using GROK.

Here is my config:

filter {
grok {
break_on_match => false
keep_empty_captures => true
match => { "message" => ["%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{DATA:datagridobject} [event-type=%{DATA:event-type}][guid=%{DATA:guid}][user-id=%{DATA:user-id}][profile=%{DATA:profile}][session-id=%{DATA:session-id}][duration=%{DATA:duration}][result-id=%{DATA:result-id}][result-count=%{DATA:result-count}]",
"%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{DATA:datagridobject} [event-type=%{DATA:event-type}][guid=%{DATA:guid}][user-id=%{DATA:user-id}][profile=%{DATA:profile}][session-id=%{DATA:session-id}][duration=%{DATA:duration}][result-id=%{DATA:result-id}][result-count=%{DATA:result-count}][text=%{DATA:text}]"
]}
}
}

I am trying to index :

Mar 23 09:49:32 WIN-12345 Web1 [event-type=XYZ][guid=abc12345][user-id=abcdefg][profile=profileName][session-id=12345][duration=01][result-id=abc123][result-count=0][text=abc]

And it never gets to 2nd MATCH and never extract "text"

Am I missing something?

Regards,
Sunny

2

Square brackets are metacharacters that need to be escaped. That's most likely your immediate problem. Also, I'd strongly advise against more than one DATA or GREEDYDATA in a single expression. You have little control over how they match.

I suggest you use the kv filter instead. Note that your fields can be parsed as key=value pairs separated by ][ with a [ prefix and a ] suffix. In other words, extract everything between the first and final bracket into a field and use the kv filter on the result (untested):

grok {
  match => [
    "message",
    "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{NOTSPACE:datagridobject} \[%{GREEDYDATA:keyvalues}\]"
  ]
}
kv {
  source => "keyvalues"
  field_split => "\]\["
  remove_field => ["keyvalues"]
}
1 Like
3

That is absolutely brilliant.!!

Thanks so much, now I don't have to create so many MATCH filters.

Cheers,
Sunny