Grok by beats agent

Elastic Stack Logstash
#1

Hello :smiley:

I have several beats entries on my logstash, but the logs I process are different, so I would like to do a grok by beat.

example log lines :

2017/07/27 18:02:37 VCS ERROR V-16-1-54031 Resource XXXXXXXXX(Owner: Unspecified, Group: XXXXXXXXX) is FAULTED on sys rsxl010101

for this log, i have this grok :
match => { "message" => "%{DATESTAMP:mytimestamp} %{WORD:tp} %{LOGLEVEL:logLevel} %{GREEDYDATA:logMessage}" }
that's ok. It can work on %{LOGLEVEL:logLevel} in kibana with filters.

But filebeat catch another file with this line :
02/11/2021 03:21:54 ANS0361I DIAG: sessSendVerb: Error sending Verb, rc: -50

My previous grok don't understand that "ANS0361I" is an %{WORD:tp}
I can't make filters on tp is ANS0361I

Do you have any advices ? :slight_smile:

#2

The problem is not with the string ANS0361I, the pattern WORD can match it without any problem, but after that grok will try to match DIAG using the pattern LOGLEVEL, which will not work because it is not a log level, so you will get an _grokparsefailure error.

Your messages are completely different, the timestamp is different, the format of the messages is different, it is better to create another grok expression to match this second format.

1 Like
#3

thank you for your reply,

Now, i have two different grok expression, one for the first file and another for the second file.

How i can say in logstash : "this grok expression is for the files /etc/var/log/messages/*.log (for example) and this pattern is for /etc/var/log/urgent/gc.log" ?