Help needed with Logstash and syslog config

Elastic Stack Logstash
1

Hello all,

First of all I have to say that I am new to ELK, in fact I have just installed it in the system.

Let me go to the point. :smiley:

I have followed the oceand digital manual to install elk in a brand new debian system (https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04) with a few modifications (basically using the latest versions and without ssl support)

Once configured, I have done three tests using this config files:
https://www.elastic.co/guide/en/logstash/master/config-examples.html

The first two tests run perfectly but when I get to the syslog one, no way.

This is my config file:


input {
  udp {
    port => 5000
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

And this is a snaptshot of the syslog data transfer

elk.png1287×702 200 KB

As it can be seen, logstash runs fine, no warnings,the system starts listening on port 5000(udp) but when i connect to the system using nc and sending the data, the data arrives but logstash does nothing

I have been looking in the logsthash logs but found nothing.

Any help will be appreciated.

Thanks a lot for your time,

2

my match for pfsense syslogs looks like this (note the square brackets):

match => [ "message", "%{SYSLOGTIMESTAMP:syslog_timestamp}: %{WORD:syslog_severity} %{SYSLOGHOST:syslog_hostname} %{SYSLOGPROG:syslog_program}:%{SPACE} %{GREEDYDATA:syslog_message}" ]
3

Thanks for the update yuphing.

I have done the change but it does not work.

I have loaded the modified config file, logstash runs fine, at least it does not give any erri, I send the info, the system gets it but logstash does nothing.

4

You have a udp input and a couple of filters, but where are your outputs? For debugging this is a common recommendation:

output {
  stdout {
    codec => rubydebug
  }
}
5

magnusbaeck is right, I usually have something like this too for output:

output {
        if !("_grokparsefailure" in [tags]) {
                elasticsearch {
                        host => "localhost"
                        cluster => "elasticsearch_cluster"
                        index => "logstash-%{type}-%{+YYYY.MM.dd}"
                }
        } else {
                file {
                        path => "/var/log/logstash/debug-%{type}.out"
                }
        }
}
6

Yeap, you are totally right, no ouptut. Have to say it, Im idiot xD, dont know how to copy and paste.

Thanks a lot guys, problem solved

Sorry for the inconvenience