Sebp/ELK with PFSense

[stinkfly]

Hi team,

I've setup sebp/ELK (https://elk-docker.readthedocs.io/), GitHub here https://hub.docker.com/r/sebp/elk/ with ELK 6.6.1

Winlogbeat and Metricbeat work ok sending from a Windows 2016 server

Syslog from PFSense router does not receive any data. However, when I use a physical Ubuntu server with Logstash (with the same conf file) and Outputting to the Elasticsearch server running on the sebp/ELK it works fine

The documentation on sebp site suggests to use Filebeat as a "forwarding agent"

Q: Why does a physical server work and why does this image require a forwarding agent. Prefer not to install filebeat on PFsense if I don't have to - just use the GUI to point to IP:port

Conf file looks like the following

---

input {
    beats { port => 5044 }
    syslog { port => 5144 }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
    date {
      match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]	
    }
  }
}

output {
elasticsearch { 
	hosts => ["localhost:9200"] 
	index => "%{[@metadata][beat]}-%{+YYYY.MM.DD}"
	document_type => "%{[metadata][type]}"	
 }
 
  stdout { codec => rubydebug }
}

[weltenwort (Felix Stürmer)]

Hi @stinkfly,

by enabling the syslog input in logstash you should be able to use the local syslog daemon on the PFSense host as the "forwarding agent". Do you mean the syslog agent does not receive any log entries from the processes on the PFSense host or logstash does not receive and log entries from the syslog agent?

I feel we require some more details about your setup in order to get a clear picture.

[stinkfly]

Hi Felix, yes I mean Logstash does not receive any log entries from PFsense

Maybe this will explain it

PFSense with syslogd package installed (not even sure this is required)

From the PFsense GUI (System -> you enter IP and Port, e.g. 192.168.0.1:5144

PFSense -> Physical server with Ubuntu 18.04, logstash - using conf file from above, works fine

PFSense -> to Logtstash container (part of sebp/ELK) - using conf file from above, does NOT work. I know there are no firewall issues as winlogbeat and metricbeat work fine

[weltenwort (Felix Stürmer)]

Thank you for providing these details. Could you also tell us how you started the container that logstash runs in?

In order to make the port 5144, which the syslog input listens on, accessible from the outside, it must be specified as an exposed port (with the correct protocol) when the container is started. For the docker run command line this would be something like -p 5144:5144/udp if pfsense uses the mechanism from RFC5426 to send the log entries.

[stinkfly]

Hi Felix, the command I run is

docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 5144:5144 -Xms512m -Xmx2g -it --restart unless-stopped --ulimit nofile=65536:65536 -v elk1-data:/var/lib/elasticsearch --name elk-01 sebp/elk

I'll try the -p 5144:5144/udp and see how it goes. Thank you