I have a 3 node setup of ELK Stack. For last few weeks I am facing a strange problem where Logstash is running without any issues but does not output any logs either to the Elasticsearch or Console. Especially with beats plugin, it was working perfect since last few days but for maintenance purpose our servers were shut down for few hours and when back it stopped working. The issue is with Beats, Cisco Syslogs and ASA, the twitter plugin is working fine.
Notes:
- I am running logstash as ROOT
- Checked all the ports whether they are listening or not using netstat -ntulp
- The default conf.d directory does not contain any other file.
Here I am posting my beats.conf file
input {
beats {
port => 5044
type => "beats"
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
filter {
if [type] == "beats" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
if [type] == "beats" {
elasticsearch {
hosts => ["10.129.137.245:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
stdout {
codec => rubydebug
}
}
}