Problem with send log from pfsense 2.2 to ELK

Elastic Stack Logstash
1

hi i install ELK with elasticsearch 1.7.1 and logstash 1.5.4 and kibana 3 and try to send my firewal logs to ELK i use pfsense 2.2.3 and i config all but have difrent error in start logstash and i got this damn error many thime
how can i fix it and using logstash
this is my logstash.conf file

   input {
    tcp {
        type => syslog
        port => 5010
    }
    udp {
        type => syslog
        port => 5010
    }

}
filter {
  if [type] == "pfsense" {
    grok {
      match => [ "message" ]
    }

    if [prog] == "filterlog" {
       grok {
         patterns_dir => "./patterns"
         match => [ "msg", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" ]
       }
       if [src_ip] {
         geoip {
           source => "src_ip"
           target => "geoip"
         }
         mutate {
           convert => [ "[geoip][coordinates]", "float" ]
           remove_tag => [ "_grokparsefailure" ]
         }
       }
    }
  }
}

output {

  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

please any help or source about config (that work really) for logstash with pfsense 2.2 ...
tankyou

2 
grok {
  match => [ "message" ]
}

This isn't correct usage of the grok filter since it's missing a grok expression to match against the `message´ field. The grok filter further down has the right form.

1 Like
3

how can i use this

not i recive new error that say pattern %{PFSENSE_LOG_DATA} not defined
i think the grok not see my pattern file
i use logstash 1.5 and i see that this is change logstash-pattern-core
how can i say to logstash to see my external pattern file and mach my field with pattern file

4

I just responded in the other thread, but please continue the discussion here since your problem is different the other thread.

1 Like
5

tank you so much for helping
i fix that error and now work and my pfsense send log correctly nad see log come in /var/log/logstash/log/studt
nut have another problem
i see logs not come flexible and their is simply and so kibana can not parse them to charts and map
this format of my logs see
i thing its not work correctly because get log but its hard to understand that logs and can not use them in geo map and charts

this my on log i receive :

<134>Aug 22 16:10:43 filterlog: 253,16777216,,1422441928,em0,match,block,in,4,0x0,,128,4132,0,none,17,udp,78,172.16.0.18,X.X.X.X,137,32,23