Differentiate Linux syslog log from Windows event log

Elastic Stack Logstash
1

Hi everyone,

we have 2 conf file in Logstash (version 6.8.6) conf.d directory to differentiate Microsoft log from Linux log, the first:

input {
       tcp {
              port => 3515
              codec => json
              }
      }
filter {
        mutate {
                 add_tag => ["forwardedevtx"]
               }
       }
output
       {
       elasticsearch {
                      hosts => [ "nodeX:9200", "nodeX:9200", "nodeX:9200" ]
                      index => "forwardedevtx-%{+YYYY.MM.dd}"
                     }
       }

the second:

input {
  tcp {
    port => 5000
    type => syslog
  }
}
filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
    mutate {
             add_tag => ["forwardedsyslog"]
           }

  }
}
output {
  elasticsearch { 
		  hosts => [ "nodeX:9200", "nodeX:9200", "nodeX:9200" ]
		  index => "forwardedsyslog-%{+YYYY.MM.dd}"
		}
}

I can't explain why in Kibana I can't see this difference, if I select forwardedsyslog or forwardedevtx index pattern I see all the logs, both Microsoft and Linux.

I would like to have into forwardedsyslog index only the Linux one, and into forwardedevtx only the Microsoft one, what am I missing?

Thanks! :slight_smile:

2

Before each output please include an if statement

if "forwardedsyslog" in [tags] {
output ...
}

1 Like
3

Hi grumo35,

thanks for your reply.
I'll give it a try and let you know :slight_smile:

4

Here are my new conf file, but nothing as changed:

input {
       tcp {
              port => 3515
              codec => json
              }
}
filter {
        mutate {
                 add_tag => ["forwardedevtx"]
               }
}
output {
  if "forwardedevtx" in [tags] {
    elasticsearch {
                    hosts => [ "nodeX:9200", "nodeX:9200", "nodeX:9200" ]
                    index => "forwardedevtx-%{+YYYY.MM.dd}"
                  }
  }
}

and:

input {
  tcp {
    port => 5000
    type => syslog
  }
}
filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
    mutate {
             add_tag => ["forwardedsyslog"]
           }

  }
}
output {
  if "forwardedsyslog" in [tags] {
    elasticsearch {
		    hosts => [ "nodeX:9200", "nodeX:9200", "nodeX:9200" ]
		    index => "forwardedsyslog-%{+YYYY.MM.dd}"
    }
  }
}

I can see all logs mixed both in forwardedsyslog and forwardedevtx index patterns into Kibana.
In addition, I wonder if we're collecting doubled logs into index patterns, both Microsoft and Linux logs are going into each different index??

Thanks :slight_smile:

5

Does this help?

1 Like
6

thanks for your reply!
I'll take a look into it!

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.