Receive logs from Pfsense but noting show in charts


(Mega Robo) #1

hi every one
i install ELK with elasticsearch and logstash 1.5 and kibana 3 and use pfsense 2.2.3 its work correctly and send logst to logstash and i can see logs in kibana and in /var/log/logstash/logstash.stutd but i created dashbourd and data can not set in this chart and i think data format not correctly

<134>Aug 24 13:43:16 filterlog: 75,16777216,,1000007870,em0,match,block,in,4,0x0,,128,5277,0,none,17,udp,78,X.X.X.X,1X.X.X.X,137,137,58

how can i fix it
log come to my logstash but its not completely and organized parsed and they are not correct for logstash and kibana
i hope can explain my problem


(Magnus B├Ąck) #2

What does your Logstash configuration look like?


(Mega Robo) #3

input {
tcp {
type => syslog
port => 5025
}
udp {
type => syslog
port => 5025
}
}
filter {
if "PFSense" in [tags] {
grok {
add_tag => [ "firewall" ]
match => [ "message", "<(?.)>(?(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?.?): (?.*)" ]
}
mutate {
gsub => ["datetime"," "," "]
}
date {
match => [ "datetime", "MMM dd HH:mm:ss" ]
}
mutate {
replace => [ "message", "%{msg}" ]
}
mutate {
remove_field => [ "msg", "datetime" ]
}
}
if [prog] =~ /^filterlog$/ {
mutate {
remove_field => [ "msg", "datetime" ]
}
grok {
patterns_dir => "/etc/logstash/patterns"
match => [ "message", "%{LOG_DATA}%
{IP_SPECIFIC_DATA}%
{IP_DATA}%
{PROTOCOL_DATA}" ]
}
mutate {
lowercase => [ 'proto' ]
}
geoip {
add_tag => [ "GeoIP" ]
source => "src_ip"
}
}
}
output {

elasticsearch { host => localhost }
stdout { codec => rubydebug }
}


(system) #4