Logstash syslog message, doesn't choose right if statement

Elastic Stack Logstash
1

Hi,

I'm trying to create a logstash pipeline for cisco FMC audit log.

I have create 3 if statements and would like for logstash to parse the syslog message according to the if statement.

Here is two example syslog messages

syslog message one
<189>Mar 7 13:09:01 ADIGCFMC01 sfdccsm: [FMC_AUDIT_LOG] ADIGCFMC01.test.net: test.user@test.net@10.10.10.25, Devices > Device Management > NGFW Interfaces, Page View

syslog message two

<189>Mar 7 10:14:25 ADIGCFMC01 sfdccsm: [FMC_AUDIT_LOG] ADIGCFMC01.test.net: csm_processes@Default User IP, Login, Login Success

Here is my logstash pipeline

input {
          udp {
            port => 1514
            type => "syslog"
          }
        }

        filter {
          if "FMC_AUDIT_LOG" in [message] and [message] !~ '/^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\s/' {
            grok {
              match => { "message" => '<%{INT:Facility}>%{SYSLOGTIMESTAMP:timestamp}%{SPACE}%{HOSTNAME:hostname}%{SPACE}%{USERNAME:TEST}:%{SPACE}\[%{USER:ciscoauth}\]%{SPACE}%{GREEDYDATA:message}'
              }
            add_tag => ["one_value"]
            } 
          } else if [message] =~ '/^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\s/' {
            grok {
              match => {
                "message" => '<%{INT:Facility}>%{SYSLOGTIMESTAMP:timestamp}%{SPACE}%{HOSTNAME:hostname}%{SPACE}%{USERNAME:process}:%{SPACE}\[%{USER:ciscoauth}\]%{SPACE}%{HOSTNAME:hostname123}:%{SPACE}%{DATA:username}%{DATA:username}%{EMAILADDRESS:email}@%{IPV4:ipv4},%{SPACE}%{GREEDYDATA:message}'
              }
              add_tag => ["two_value"]
            }
          } else {
            drop {}
          }

          date {
            match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
          }

The syslog message always chooses to run with the first if-statement and ignore the second one.

Request:
I would like logstash to use the first if statement if the syslog message contains FMC_AUDIT_LOG and an email address and the second if statement if the syslog message contains FMC_AUDIT_LOG but not the email address.

Question:

  1. How can I change the if statement so it chooses the right if statement to use?
  2. Can this be done another way that is much easier than using if statements and grok?

Thanks.

2

Your messages start with <, so they are never going to match that pattern. Thus they will always match the first if.

I would suggest using dissect for the fixed prefix, then use grok for the rest of the line. See here for an example.

Note that if you do not set the overwrite option and your output field matches an existing field (e.g. [message]) then you will end up with that field being an array.

Note also that there are standard grok patterns that can be used to pick out an email address, and that recognizing an email address is a ridiculously complex problem if you need to support international addresses.

3

Hi @Badger

Thank you for the input.

I used your information and now the syslog message is parsing correctly.

The if statement is working and I had some issue with the last part of the message but solved it with the below code.

I split the last part of the message and created new fields with it.

mutate {
            split => { "data" => ", "}

            add_field => { "[event][type]" => "%{[data][1]}" }
            add_field => { "[event][dataset]" => "%{[data][2]}" }
          }

Thanks!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.