A question setting up logstash with raspberry pi


#1

Hi all,

I am setting up a raspberry pi 3 to send syslogs (rsyslog) to a server running logstash. I have installed filebeat on the raspberry pi in order to ship the logs to the logstash server. For some reason, the logstash server is not seeing these logs. The Raspberry pi is running debian stretch 9.1.

I installed the raspberry pi binary via https://beats-nightlies.s3.amazonaws.com/jenkins/filebeat/ and created some systemd files
On raspberry pi, here is filebeat config: The Cert was created on the logstash server(10.100.1.7:5044)

filebeat:
  prospectors:
    -
      paths:
        - /var/log/auth.log
        - /var/log/syslog
      #  - /var/log/*.log

      input_type: log

      document_type: syslog

  registry_file: /var/lib/filebeat/registry

output:
  logstash:
    hosts: ["10.100.1.7:5044"]
    bulk_max_size: 1024

    tls:
      certificate_authorities: ["/etc/ssl/logstash-forwarder.crt"]

shipper:

logging:
  files:
    rotateeverybytes: 10485760 # = 10MB

Here is config files on logstash server:

02-beats-input.conf

input {
 beats {
   port => 5044
   host => "0.0.0.0"
   ssl => true
   ssl_certificate => "/etc/ssl/logstash-forwarder.crt"
   ssl_key => "/etc/ssl/logstash-forwarder.key"
   congestion_threshold => "40"
  }
}

10-syslog-filter.conf

filter {
  if [type] == "syslog" {
grok {
  match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
  add_field => [ "received_at", "%{@timestamp}" ]
  add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
  match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
  }
}

And the other config file: The host ip below is the Elastic search running on another server(10.100.1.16)
30-elasticsearch-output.conf

output {
  elasticsearch {
    hosts => ["10.100.1.16:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

sudo /opt/logstash/bin/logstash --configtest -f /etc/logstash/conf.d/

returns OK

Any help would be appreciated!


(Magnus B├Ąck) #2

Look in the logs of both Filebeat and Logstash. Check the network connectivity from the RaspberryPi to the Logstash box.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.