Help grokking a syslog

Elastic Stack Logstash
1

Hello, i need some help to grok this. Should i use the CEF pattern or just manually hack away at my own? I cant seem to find any built-in patterns that work..

The message looks like part of it is syslog, part is key=value pairs, so i am just unsure..

<143>Nov 29 08:53:20 10.253.0.20 CEF: 0|Aruba Networks|ClearPass|6.6.8.100017|2001|Failed Authentications|5|duser=oscar.mainen destinationServiceName=Company - Aruba 802.1X Wireless dpriv=[User Authenticated] cs1=AD:server01.domain.net cs1Label=Authentication Source requestMethod=EAP-PEAP cs2=UNKNOWN cs2Label=System Posture Token outcome=Company - Aruba 802.1X Wireless Update Endpoint Location, Company - Aruba 802.1X Wireless Profile - AD Authenticated User dmac=78xxxxxxxec src=10.253.0.44 cn1=0 cn1Label=Error Code rt=Nov 29 2017 08:52:01 dvc=10.253.0.20 cat=Session Logs
2

What is the expected output?

As you rightly mentioned, it will be a combination of KV and grok. Also, you might have to write your own pattern if you want specific fields out of the given input.

3

ok great, what would an example of using different filters against the same message?

4

Here is an excerpt of a conf i had written once:

filter {  if "getproxy" in [tags] {

    grok {
            patterns_dir => ["/etc/logstash/patterns"]
            match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{IP:srcip} %{PROXYBASE:base} %{OUBASE:oubase} %{GREEDYDATA:msg}" }
            add_tag => [ "User_Present" ]
            remove_tag => ["getproxy"]
            tag_on_failure => [ "unparsed" ]
    }
    grok {
            patterns_dir => ["/etc/logstash/patterns"]
            match => { "oubase" => "%{OU:ou}com\/%{GREEDYDATA:suser}" }
            tag_on_failure => [ "unparsed" ]

            add_field =>  {"user.source" => "%{suser}"}
            remove_field => ["suser"]
            remove_field => ["oubase"]
    }

    if "unparsed" in [tags] {
            grok {
                    patterns_dir => ["/etc/logstash/patterns"]
                    match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{IP:srcip} %{PROXYBASE:base} %{GREEDYDATA:msg}" }
                    add_tag => ["User_Absent"]
                    remove_tag => ["unparsed", "getproxy"]
                    tag_on_failure => ["unparsed"]
            }
    }

    if "unparsed" not in [tags] {
            kv {
                    source => "msg"
                    value_split => "="
                    target => "msg"
            }
            kv {
                    source => "base"
                    value_split => "="
                    target => "base"
            }

            mutate {
                            add_field =>  {"ip.device" => "%{srcip}"}
                            remove_field => ["srcip"]
                            remove_field => ["product", "product_version", "user"]
            }

    }

    date {
            match => [ "timestamp", "MMM dd HH:mm:ss" ]
    } } }
5

Hi @NerdSec, so the idea is to start with a "loose" filter, then continue filter with more and more detail until you capture every possible scenario?

6

@NerdSec can i have a copy of your input message, so that i can work out how this all works?

7

Hi ajhstn,

I have actually copied this from an earlier question I had posted on the forum. I do not have access to the system as of now which had the input logs.

I can help you with any issues you might have with your input!

Regards,
N

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.