Learning Grok

[Jukocross]

Hi, i'm learning grok and trying out to craft my own filter.
Below is the example which i have tried using https://grokdebug.herokuapp.com to debug.
However, i still received grokparsefailure when output to elasticsearch

Original Log:
"<44>Original Address=192.168.218.133 This is a test message generated by Kiwi SyslogGen"

Following grok filter:

filter {
 grok {
   match => { "message" => "%{SYSLOG5424PRI}%{GREEDYDATA:msgtype}=%{IP:source} %{GREEDYDATA:msg}"}
   add_tag => [ "Kiwi_Gen_Tag" ]
 }
}

[Krunal_kalaria]

Hey @Jukocross,

Can you just add one more space in last braces following is your grok
match => { "message" => "%{SYSLOG5424PRI}%{GREEDYDATA:msgtype}=%{IP:source} %{GREEDYDATA:msg}"}

Now in end of the line this sentence is their %{GREEDYDATA:msg}"} so put the space in last like %{GREEDYDATA:msg}" }

So your full grok look like,

match => { "message" => "%{SYSLOG5424PRI}%{GREEDYDATA:msgtype}=%{IP:source} %{GREEDYDATA:msg}" }

Try this bold one it may be worked for you!

Thanks & Regards,
Krunal.

[Jukocross]

Hi, Thanks for the advise, I have edit accordingly but the error still occur. Is there any method to debug such error?

[Christian_Dahlqvist]

Have a look at this blog post, which shows how to work with Logstash when developing config. This blog post on writing efficient grok config is also very useful.

[verboese (Jan D.)]

Hi.
I don't think "SYSLOG5424PRI" is known by logstash (at least not with my 5.6). Try to define a pattern file with definition of SYSLOG5424PRI ...

[Jukocross]

Hi, i have remove "SYSLOG5424PRI" but the kiwi gen is generating new format of logs hence i have make some changes below. However, i still faced with the error and hope could get some advise on it.

Log: "<44> Jun 8 10:36:42 TestClient-PC SyslogGen This is a test message generated by Kiwi SyslogGen"

filter {
 grok {
   match => { "<%{NUMBER:priority}>%{CISCOTIMESTAMP:logtime} %{WORD:hostname}-%{WORD:device} %{WORD:origin} %{GREEDYDATA:msg}" }
   add_tag => [ "Kiwi_Gen_Tag" ]
   tag_on_failure => []
 }
 mutate { ... }
 translate { ... }
}

Even with tag_on_failure, i still have the _grokparsefailure tag. However, all the field have translated as i wanted.