Constraint: logs are coming in collected by Splunk, forwarded, along with many other log sources, to logstash. So no winlogbeat, no logstash-input-eventlog plugin. We have to pick out the Windows events from the flood of logs, and parse them in logstash. Picking them out is easy enough. However, parsing them seems problematic because the Windows log format is wonky. The first twelve or so fields are nicely formatted with fieldname=value, then it starts going to fieldname:value.
Surely someone has had to parse these events in logstash before right? Can you share the filter you used?
Here is an example log
"<13> dc01 08/11/2020 10:58:31 AM\n\tLogName=Security\n\tSourceName=Microsoft Windows security auditing.\n\tEventCode=4624\n\tEventType=0\n\tType=Information\n\tComputerName=dc01.acme.local\n\tTaskCategory=Logon\n\tOpCode=Info\n\tRecordNumber=4796622\n\tKeywords=Audit Success\n\tMessage=An account was successfully logged on.\r\n\t\Subject:\r\n\tSecurity ID:S-1-0-0\r\n\tAccount Name:-\r\n\tAccount Domain:-\r\n\tLogon ID:0x0\r\n\tLogon Information:\r\n\tLogon Type:3\r\n\tRestricted Admin Mode:-\r\n\tVirtual Account:No\r\n\tElevated Token:Yes\r\n\t\Impersonation Level:Impersonation\r\n\t\r\n\tNew Logon:\r\n\tSecurity ID:S-1-5-21-2344382208-301227701-2769393404-17886\r\n\tAccount Name:svc-acme\r\n\tAccount Domain:acme.LOCAL\r\n\tLogon ID:0x88AF090D\r\n\tLinked Logon ID:0x0\r\n\tNetwork Account Name:-\r\n\tNetwork Account Domain:-\r\n\tLogon GUID:{25332F90-F62B-A867-DC37-689B2B97A186}\r\n\t\r\n\tProcess Information:\r\n\tProcess ID:0x0\r\n\tProcess Name:-\r\n\t\Network Information:\r\n\tWorkstation Name:-\r\n\tSource Network Address:10.1.1.12\r\n\tSource Port:53477\r\n\t\r\n\tDetailed A",
I first tried
#Delete \t and \r from messages
mutate{
gsub => ["message", "\t", ""]
}
kv{
source => "message"
value_split => "="
field_split => "\n"
}
and that worked good for the first 12 fields, but then I am left with a bunch of unparsed items, separated by \r and :'s. I have no idea if I can parse them again using another mutate. Thoughts?