Distinguish between syslog inputs

[kernelpanic (FreeBSD user)]

Hello, can anyone tell me how I would distinguish between two different syslog sources in Logstash, for example one source being a firewall device and another being an ESXi host? I know you can use tags but as I can't install anything at the source then I can't place tags on the message before they're sent to Logstash.

Thanks for any help.

[paz]

Hi,

There are a couple of ways you can do that.

1. Have a separate Logstash input for each source (different ports) and tag/add fields on each source respectively.
2. Filter based on the "host" field that should be available and contain the originating IP/hostname, if you know them beforehand.

More options could be available depending on the setup, but those above should be the most straightforward.

[kernelpanic (FreeBSD user)]

Thankyou, I'll try filtering on the host / IP address field.

[kernelpanic (FreeBSD user)]

Hello again, I did try to add the syslog configuration to my logstash.conf but instead of accepting syslog messages it threw the Windows event logs I was also collecting into the syslog index. Can anyone tell me where I've gone wrong? My configuration is below:

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["192.168.56.226:9200", "192.168.52.251:9200", "192.168.52.252:9200"]
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM}"
    document_type => "%{[@metadata][type]}"
  }
}
#########################################################################################
#########################################################################################
input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  elasticsearch {
       hosts => ["192.168.56.226:9200", "192.168.52.251:9200", "192.168.52.252:9200"]
    index =>["syslog-%{+YYYY.MM}"]
  }
}

[kernelpanic (FreeBSD user)]

I think I've sorted it with the following config:

input {
  beats {
    port => 5044
    type => wineventlog
  }
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}


filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}



output {
  if [type] == "wineventlog" {
   elasticsearch {
     hosts => ["192.168.56.226:9200", "192.168.52.251:9200", "192.168.52.252:9200"]
     manage_template => false
     index => "%{[@metadata][beat]}-%{+YYYY.MM}"
     document_type => "%{[@metadata][type]}"
 }
}

  if [type] == "syslog" {
   elasticsearch {
    hosts => ["192.168.56.226:9200", "192.168.52.251:9200", "192.168.52.252:9200"]
    index => ["syslog-%{+YYYY.MM}"]
   }
  }
 }