I was studying to retrieve some detailed information from Windows Event to find out logon failure records. Those detailed information is located in field "message" or "InsertionStirng".i.e. Account Domain, Account Name and etc.. Please see the below sample (in JSON format)
{ "host":"twLab-USVMDC01", "Logfile":"Security", "message":"An account failed to log on.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-0-0\r\n\tAccount Name:\t\t-\r\n\tAccount Domain:\t\t-\r\n\tLogon ID:\t\t0x0\r\n\r\nLogon Type:\t\t\t3\r\n\r\nAccount For Which Logon Failed:\r\n\tSecurity ID:\t\tS-1-0-0\r\n\tAccount Name:\t\tustest1\r\n\tAccount Domain:\t\tus-teset\r\n\r\nFailure Information:\r\n\tFailure Reason:\t\t%%2313\r\n\tStatus:\t\t\t0xc000006d\r\n\tSub Status:\t\t0xc000006a\r\n\r\nProcess Information:\r\n\tCaller Process ID:\t0x0\r\n\tCaller Process Name:\t-\r\n\r\nNetwork Information:\r\n\tWorkstation Name:\tTWLAB-POWERSHEL\r\n\tSource Network Address:\t10.1.93.115\r\n\tSource Port:\t\t60633\r\n\r\nDetailed Authentication Information:\r\n\tLogon Process:\t\tNtLmSsp \r\n\tAuthentication Package:\tNTLM\r\n\tTransited Services:\t-\r\n\tPackage Name (NTLM only):\t-\r\n\tKey Length:\t\t0\r\n\r\nThis event is generated when a logon request fails. It is generated on the computer where access was attempted.\r\n\r\nThe Subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\r\n\r\nThe Logon Type field indicates the kind of logon that was requested. The most common types are 2 (interactive) and 3 (network).\r\n\r\nThe Process Information fields indicate which account and process on the system requested the logon.\r\n\r\nThe Network Information fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\r\n\r\nThe authentication information fields provide detailed information about this specific logon request.\r\n\t- Transited services indicate which intermediate services have participated in this logon request.\r\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\r\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.", "Category":12544, "ComputerName":"twLab-USVMDC01.us.test.local", "EventType":"audit_failure", "RecordNumber":2847012, "SourceName":"Microsoft-Windows-Security-Auditing", "TimeGenerated":"2016-01-12 12:19:48 +0800", "TimeWritten":"2016-01-12 12:19:48 +0800", "Type":"audit_failure", "User":null, "@version":"1", "@timestamp":"2016-01-12T04:20:02.843Z", "type":"winevent", "EventID":4625, "Metadata":[ "S-1-0-0", "-", "-", "0x0", "S-1-0-0", "ustest1", "us-teset", "0xc000006d", "%%2313", "0xc000006a", "3", "NtLmSsp ", "NTLM", "TWLAB-POWERSHEL", "-", "-", "0", "0x0", "-", "10.1.93.115", "60633" ], "tags":[ "_grokparsefailure" ] }
I tried to leverage kv filter to parse message field but got failure and don't know how to proceed. Also, I tried to leverage grok filter and verified in online grok constructor (http://grokconstructor.appspot.com/do/match), but it still failed in Logstash. Here are the filters that I wrote, please hint me anything should be changed to make it. Thank you.
With kv filter
filter { kv { field_split => "\r\n" value_split => ":" prefix => "arg_" } }
With grok filter
grok { match => { 'InsertionStrings' => '\[\"%{DATA:Subject_SecurityID}\",\"%{DATA:Subject_AccountName}\",\"%{DATA:Subject_AccountDomain}\",\"%{DATA:Subject_LogonID}\",\"%{DATA:AccountForWhichLogonFailed_SecurityID}\",\"%{DATA:AccountForWhichLogonFailed_AccountName}\",\"%{DATA:AccountForWhichLogonFailed_AccountDomain}\",\"%{DATA:FailureInformation_Status}\",\"%{DATA:FailureInformation_FailureReason}\",\"%{DATA:FailureInformation_SubStatus}\",\"%{DATA:LogonType}\",\"%{DATA:DetailedAuthenticationInformation_LogonProcess}\",\"%{DATA:DetailedAuthenticationInformation_AuthenticationPackage}\",\"%{DATA:NetworkInformation_WorkstationName}\",\"%{DATA:ProcessInformation_CallerProcessName}\",\"%{DATA:DetailedAuthenticationInformation_TransitedServices}\",\"%{DATA:DetailedAuthenticationInformation_KeyLength}\",\"%{DATA:ProcessInformation_CallerProcessID}\",\"%{DATA:DetailedAuthenticationInformation_PackageName_NTLMonly}\",\"%{DATA:NetworkInformation_SourceNetworkAddress}\",\"%{DATA:NetworkInformation_SourcePort}\"\]' } }