After I successfully integrated the logs from my Synology. I now go over to my pfsense. There a some tutorials out in the www. Unfortunately none of them is working properly or just with old versions.
I'm following the tutorial from: http://pfelk.3ilson.com/
At the moment I recieve the logs in the ELK stack but not parsed correctly. The biggest problem is probably the eventid.
filter {
if "PFSense" in [tags] {
grok {
add_tag => [ "firewall" ]
match => [ "message", "<(?<evtid>.*)>(?<datetime>(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?<prog>.*?): (?<msg>.*)" ]
}
mutate {
gsub => ["datetime"," "," "]
}
date {
match => [ "datetime", "MMM dd HH:mm:ss" ]
timezone => "America/New_York"
}
mutate {
replace => [ "message", "%{msg}" ]
}
mutate {
remove_field => [ "msg", "datetime" ]
}
}
if [prog] =~ /^filterlog$/ {
mutate {
remove_field => [ "msg", "datetime" ]
}
grok {
patterns_dir => "/etc/logstash/conf.d/patterns"
match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}",
"message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" ]
}
mutate {
lowercase => [ 'proto' ]
}
geoip {
add_tag => [ "GeoIP" ]
source => "src_ip"
# Optional GeoIP database
# Comment out the below if you do not wise to utilize and omit last three steps dealing with (recommended) suffix
database => "/etc/logstash/GeoLite2-City.mmdb"
}
}
}
I made a picture from the view in Kibana at the moment. I tried with: BASE10NUM but that doesn't work.
Can someone help me. This is going to be more difficult I think. But I have all the grok patterns from the tutorial and config. Maybe I just have to make a few adjustments.