Grok regex to filter custom data


(Radu Malica) #1

Hi

I am trying to define a grok to parse the following line: (It's from an Alcatel Lucent 7210SAS device)

Jun 21 12:10:49 10.20.30.40 TMNX: 7655 Base SYSTEM-WARNING-ssiSyncConfigOK-2012 [Syncronization of Config files]: Configuration files have been successfully syncronized

or

Jun 21 12:10:49 10.20.30.40 TMNX: 7655 Base USER-MINOR-cli_user_logout-2002 [admin]: User from 10.23.44.22 logged out

On the above lines, SYSTEM-WARNING and USER-MINOR will be always 2 words delimited by hyphen, cli_user_logout or ssiSyncConfigOK must be taken as a whole word, and 2012 or 2002 as integer or string.

[admin] or [syncronization of config files] must be taken also as a whole string which is the action let's say.

I compiled this in grok debugger for the second line but in logstash doesn't work:

(?^(.?-.?))-(?(.*)) (?[([^[]]+)]): %{GREEDYDATA:message_log}

Which in grokdebugger correctly outputs "USER-MINOR", "2002" and "cli_user_logout-2002" plus the rest of the message.

I figured out the part with the date, IP and static word plus the sequence of log file "7655" which is always different but i am stuck on the next message

I would like a line which can match all of the above in both cases let's say a generic one will look like:

Jun 21 12:10:49 10.20.30.40 TMNX: 7655 Base SEVERITY-TYPE-event-eventid [action]: User from 10.23.44.22 logged out

SEVERITY-TYPE is always upper case and 2 word with hyphen in the middle sentence
event is always a whole world regardless if it has any punctuation or not inside.

[action] is always between square brackets and data must be filtered without the brackets, only the value

I've had a hard time making it work in grokdebugger site, but wasn't able to do a generic one, just for the values with "_" in event. The above grok outputs this:

{
"severity": [
"USER-MINOR"
],
"id": [
"cli_user_logout-2002"
],
"user": [
"[admin]"
],
"message_log": [
"User from 10.23.44.22 logged out"
]
}

but this doesn't work in logstash...

Appreciate any help

Thanks


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.