Learning grok filter


(Dao Quang Truong) #1

Hi everyone, im newbie with Elastic Stack
Following to https://grokdebug.herokuapp.com/ i can create very simple grok filter like
2017-08-28T04:34:32.651Z ABC 123123 671 => %{TIMESTAMP_ISO8601} %{WORD} %{NUMBER} %{NUMBER}
But when i have some different kinh of log message i cant filter by my own.
Like [2017-08-28T04:34:32.651Z][ABC][123123] 671 or many kind.
So my question is how can i understand correctly all the syntax in Grok debugger patterns.
I have read but not understand all of it
Thanks


(Dao Quang Truong) #2

Can anyone help me
Thanks!!!!


(Brandon Hatch) #3

That is a bit of an open ended question. Are you familiar with regex? GROK is basically just a simpler way of using prebuilt regex statements. You can see the full patterns here.

The biggest thing I learned with Grok is that you are not required to have every single field be a GROK pattern.
So look at this log line here.
Web01 collected 5 events

You can use a grok pattern like this:
^%{WORD:serverName} collected %{INT:numberOfEvents} events$
This will then give you results of
{"serverName":"Web01","numberOfEvents":5}
Notice that I used a combination of standard text, grok patterns and regular regex. GROK basically just replaces the grok pattern with the corresponding regex pattern. Any part of the text that isn't explicitly named just gets ignored.

So taking that knowledge lets apply it to your test data above. Lets use this grok statement.
^\[%{TIMESTAMP_ISO8601:timestamp}\]\[%{WORD:word1}\]\[%{INT:number1}\] %{INT:number2}$
This gives a result of:
{"timestamp":"2017-08-28T04:34:32.651Z","word1":"ABC","number1":123123,"number2":671}

Notice that I had to escape the brackets as brackets are a special character in REGEX.

I personally like this site for testing patterns.
http://grokconstructor.appspot.com/do/match


image


(Dao Quang Truong) #4

Hi @bhatch
Thank for reply.
I dont know the regex, that why i cant understand the grok filter.
Now i can create my own, Thanks


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.