Filtering Windows Event Logs

Elastic Stack Logstash
1

Hi,

I am creating a POC of ELK for analysing windows event logs. I am not getting how to apply filters on these event logs. Is there any pattern defined to be directly used in the Grok filter like for Syslogs?

If not then, how can I define my own regex for the event logs and use them in the logstash filter?

2

How are you getting the events into Logstash? Winlogbeat?

3

Hi Magnus,

I am getting the events through tcp port into the logstash shipper and then through a MQ(redis) into the Logstash indexer.

4

And what's sending the events via TCP? What I'm really getting at is what do the events currently look like? Output from a stdout { codec => rubydebug } output would be useful.

5

Hi Magnus,

I get windows event log as (If I am not wrong it is snare syslog format)

Test_Host MSWinEventLog 0 Security 3027 Fri May 24 09:30:43 2013 593
Security Administrator User Success Audit LE5678WSP Detailed
Tracking A process has exited:Process ID: 656 User Name:
Administrator Domain: LE5678WSP Logon ID: (0x0,0x6C52)

6

I get windows event log as (If I am not wrong it is snare syslog format)

You're getting Windows events over syslog? What's sending that?

Please show output from a stdout { codec => rubydebug } output.

7

It's a TIBCO product CLMS which is sending the logs

8

Hi Magnus,

Here is the output from stdout { codec => rubydebug }

<13>Aug 23 17:58:25 ::ffff:123.123.123.123 Aug 3 17:58:26 123.123.123.123 MSWinEventLog 0 Security 0 Tue Aug 23 12:28:20 2012 4776 Microsoft-Windows-Security-Auditing Unknown Success Audit domainname.com Credential Validation The computer attempted to validate the credentials for an account. Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon Account: username Source Workstation: workplacename Error Code: 0x0 14307364
{
"message" => "<13>Aug 23 17:58:25 ::ffff:123.123.123.123 Aug 3 17:58:26 123.123.123.123 MSWinEventLog\t0\tSecurity\t0\tTue Aug 23 12:28:20 2012\t4776\tMicrosoft-Windows-Security-Auditing\t\tUnknown\tSuccess Audit\tdomainname.com\tCredential Validation\t\tThe computer attempted to validate the credentials for an account. Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon Account: username Source Workstation: workplacename Error Code: 0x0\t14307364",
"@version" => "1",
"@timestamp" => "2016-09-20T05:28:36.438Z",
"host" => "ip-10-0-0-10",
"tags" => [
[0] "_grokparsefailure"
],
"wins_client_ip" => "123.123.123.123"
}

The configuration used is

input {
stdin {}
}
filter {
grok {
match => {"message" => "%{SYSLOGBASE}"}
}
grok {
match => {"message" => "%{IPV4:wins_client_ip}" }
}
}
output {
stdout {
codec => rubydebug
}
}

9

Okay. I don't have time to help you fix the grok expression, but once you extract the timestamp and the other initial fields you should be able to use a csv filter to split the remaining string on the tab characters.

10

Hi Magnus,

I tried out csv filter but is it not going with the logs as I am not getting fields as per my need. So now I am trying to create custom patterns. I have tried out something like this

ACCNAME (?=.*Account Name:\s\w+\s)

by creating a new pattern file.

I then called it inside the config file patterns_dir => ["/opt/logstash/patterns"]

grok {
patterns_dir => ["/opt/logstash/patterns"]
match => {"message" => "%{ACCNAME:accName}" }
}

But I am not getting any field for this. This regex is working otherwise.

Is this the correct way of calling the custom patterns or is it the deprected one? Is there any other way for this?

11

I tried out csv filter but is it not going with the logs as I am not getting fields as per my need.

If you show us what you tried and what the results were we can help you.

ACCNAME (?=.*Account Name:\s\w+\s)

There's no "Account Name" in the message.