Beginner question grok + pattern matching + different line format


(goudal) #1

Hello,
I'm new to elk and I'm trying to figure out how the grok match keyword works.
What I wonder is if I want to parse different line format, do I have to put multiple grok filters one after the other one and the first match that is ok will be used ? Or do I have to first filter my line format with a test and than apply a match keyword ?
In other word does match work as a pattern-matching selector or not ?

My aim is to analyse logs from different applications which have very different formats,

THanks in advance.


#2

What you can do is the following:
Image you have 3 different patterns:

filter{
  if ("SUCCESS" not in [tags]) {
    grok{
      match => {"message" => "PATTERN1"}

      add_tag => ["SUCCESS"]
      remove_tag => ["_grokparsefailure"]
  }

  if ("SUCCESS" not in [tags]) {
    grok{
      match => {"message" => "PATTERN2"}

      add_tag => ["SUCCESS"]
      remove_tag => ["_grokparsefailure"]
  }

  if ("SUCCESS" not in [tags]) {
    grok{
      match => {"message" => "PATTERN3"}

      add_tag => ["SUCCESS"]
      remove_tag => ["_grokparsefailure"]
  }
}

And then in ouput, only push if the message does not have a _grokparsefailure tag.

It might also be possible to do this by using multiple patterns in one grok filter and by setting break_on_match (or something like that) to true. But as I don't use it, I cannot guide you for this method. (I personnaly prefer splitting them, it's longer, but easier to read and to split them into multiple .conf files)


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.