Syslog forwarding question -> Logstash (very basic)

Elastic Stack Logstash
1

Hi there,

I'm a newbie.

I'd like to forward syslog messages to my ELK stack. So basically am I right to assume logstash is capable of receiving syslog messages and parsing them without sending to a syslog server first?

I forward syslog directly from my Cisco switch, remote log to ELK server ip UDP 5514.

On my ELK server:
udp 0 0 0.0.0.0:5514 0.0.0.0:*

My input file:

input {
udp {
port => 5514
type => "syslog"

my filter file:

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGLINE}" }
}
date {
match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}

My output file:

output {
  elasticsearch {
      hosts => ["http://localhost:9200"]
      index => "syslog-%{+YYYY.MM.dd}"
      document_type => "system_logs"
  }
  stdout { codec => rubydebug }
}

when I search logstash log for incoming UDP:

[INFO ][logstash.inputs.udp ] Starting UDP listener {:address=>"0.0.0.0:5514"}

So basically input logstash UDP 5514 and output to elasticsearch 9200.

Looks fine.

kibana runs on http://localhost:5601

But doesn't seem to index my syslog messages. Is there a way to check if syslog messages are coming in at all?

2

Hey Erik,

You're right that Logstash can receive syslog messages directly, using the syslog input, or tcp/udp inputs.

If you want to check stuff is definitely getting through the front door you could try the below on your Logstash nodes:

tcpdump -A -i any dst port 5514

Enabling additional logging on your UDP input might also give some visibility: https://www.elastic.co/guide/en/logstash/current/logging.html#_logging_apis

3

You already had this block which displays any events to the console (given that you are running Logstash from console) in additional to trying to index to ES, so if you don't see anything from the console, the logs are not arriving at Logstash.

4

Hi Michael, thanks for your answer! I tried to see with TCPDUMP and syslog messages are coming in! What would be the next step?

I edited to debug this lines:

"logstash.inputs.udp" : "DEBUG",
"logstash.outputs.elasticsearch" : "DEBUG",
"logstash.outputs.stdout" : "DEBUG",

But where am I able to find the debug info?

5

Well actually I ran logstash --debug too turn all debug on.
Actually it looks fine. What else could I check?

6

When I run; /usr/share/logstash/bin/logstash --path.settings /etc/logstash --debug
no syslog messages are appearing on the console.. so must be something wrong with input ?

7

When I run: [root@host-l01 ~]# curl 'localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
close .watcher-history-7-2018.04.11 01Z34Tk8SoCWrrQj_oFlYA
green open .triggered_watches dm4mpxy_Q4GTwLJPVjQ7ng 1 0 0 0 15.5kb 15.5kb
green open .watches n0HQZ2pKT4GTvfMHrOeo2g 1 0 6 0 32.9kb 32.9kb
green open .monitoring-es-6-2018.04.11 GWH5uCuOQjGjdSkf22ZoUw 1 0 1839 12 1mb 1mb
green open .security-6 ZXbp_DODSouFZamobe3Wdg 1 0 3 0 9.8kb 9.8kb
green open .monitoring-alerts-6 whBL9bysR7au2_1bBSMtPQ 1 0 1 0 6.1kb 6.1kb

it doesn't seem to display an index from logstash or something that is increasing in values.

8

Got it to work. Apparently there was a syntax error in my input file..
corrected it, now all logstash and syslog indices show up.. :slight_smile:

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.