Hi all,
I am setting up a raspberry pi 3 to send syslogs (rsyslog) to a server running logstash. I have installed filebeat on the raspberry pi in order to ship the logs to the logstash server. For some reason, the logstash server is not seeing these logs. The Raspberry pi is running debian stretch 9.1.
I installed the raspberry pi binary via https://beats-nightlies.s3.amazonaws.com/jenkins/filebeat/ and created some systemd files
On raspberry pi, here is filebeat config: The Cert was created on the logstash server(10.100.1.7:5044)
filebeat:
prospectors:
-
paths:
- /var/log/auth.log
- /var/log/syslog
# - /var/log/*.log
input_type: log
document_type: syslog
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["10.100.1.7:5044"]
bulk_max_size: 1024
tls:
certificate_authorities: ["/etc/ssl/logstash-forwarder.crt"]
shipper:
logging:
files:
rotateeverybytes: 10485760 # = 10MB
Here is config files on logstash server:
02-beats-input.conf
input {
beats {
port => 5044
host => "0.0.0.0"
ssl => true
ssl_certificate => "/etc/ssl/logstash-forwarder.crt"
ssl_key => "/etc/ssl/logstash-forwarder.key"
congestion_threshold => "40"
}
}
10-syslog-filter.conf
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
And the other config file: The host ip below is the Elastic search running on another server(10.100.1.16)
30-elasticsearch-output.conf
output {
elasticsearch {
hosts => ["10.100.1.16:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
sudo /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/
returns OK
Any help would be appreciated!