Parsing a syslog log

Elastic Stack Logstash
#1

Im receiving the following log (some things changes for privacy, althought the password is literally sent as asterisks)

<134> 2021/03/18:13:33:41 SYSTEMNAME 0-PPE-0 : default GUI CMD_EXECUTED 1324333 0 : User invaliduser - Remote_ip 192.168.23.1 - Command "login invaliduser """ - Status "ERROR: Invalid username or password"
<134> 2021/03/18:13:33:49 SYSTEMNAME 0-PPE-0 : default GUI CMD_EXECUTED 1324393 0 : User validuser - Remote_ip 192.168.23.1 - Command "login validuser """ - Status "Success"

I

So I need to parse this as the following

ID = <134>
Timestamp = 2021/03/18:13:33:41
Hostname = SYSTEMNAME
Interface = 0-PPE-0
Section = default
Via= GUI
Action = CMD_EXECUTED
CommandCode = 1324333
CommandCodeTwo = 0
User = invaliduser
RemoteIP = 192.168.23.1
Command = login invaliduser "********"
Status = ERROR: Invalid username or password

How can I do this? What should I use?

#2

I would suggest using dissect. If you cannot dissect it then use grok.

#3

I actually attempted to use dissect but it didnt work out:

dissect {
      mapping => {
        "message" => "%{id} %{ts}  %{hostname} %{interface} : %{typeofaction} %{code} 0 :  User %{user} - Remote_ip %{remoteip} - Command "login %{user} "********"" - Status %{description}"
      }
    }
#4

You appear to have two spaces after %{ts}, but not in the actual message. Also ": %{typeofaction} %{code} 0 :" does not match ": default GUI CMD_EXECUTED 1324393 0 :". Perhaps ": %{typeofaction} %{code} %{+code} 0 :" so that [code] picks out both GUI and CMD_EXECUTED. Also, the login field pattern looks wrong.

In dissect, a pattern like "%{foo} %{bar}" matches anything that is not a space, followed by a single space, followed by anything that is not a space. When space is a delimiter neither foo nor bar can contain a space.

#5

Thank you.

Let me take another look at it and see if maybe that is what is wrong.

I went ahead and took a shot with grok as well:

filter { 
 grok { match => [ "message" => "<%{BASE10NUM:ID}> %{YEAR:Year}/%{MONTHNUM:Month}/%{MONTHDAY:Day}:%{HOUR:Hour}:%{MINUTE:Minute}:%{SECOND:Second} %{HOSTNAME:Hostname} %{DATA:Interface} : %{DATA:Type} %{DATA:CliOrGuiType} %{DATA:Action} %{INT:EventID} %{INT:ExitCode} : User %{USERNAME:User} - Remote_ip %{IP:Remote_IP} - Command "%{DATA:Command} """ - Status "%{DATA:Message}"" ] 
 }
}

It gives a error on that grok line when trying to load the conf

#6

That should be

grok { match => { "message" => '<%{BASE10NUM:ID}> ... "%{DATA:Message}"' } }

rather than

grok { match => [ "message" => "<%{BASE10NUM:ID}> ... "%{DATA:Message}"" ] }
#7

I attempted with the "{" and the "[" as well......

I do notice that you use ' instead of " ; Does that make a difference? I am escaping with \"

Maybe there lies my issue?

#8

The match option of grok expects a hash. Generally, logstash is will to convert

match => [ "fieldName", "pattern" ]

to

match => { "fieldName" => "pattern" }

and vice versa when an option expects an array. However,

match => [ "fieldName" => "pattern" ]

is neither fish nor fowl and logstash will object to it.

If pattern contains double quotes you can either surround it with single quotes or escape the double quotes using backslash.

What error is logstash logging?