# PFsense 2.3.3 and ELK **URL:** \ **Category:** Logstash\ **Created:** [April 27, 2017, 6:51pm UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928 "2017-04-27T18:51:16Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Post date:** [April 27, 2017, 6:51pm UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/1 "2017-04-27T18:51:16Z")
After I successfully integrated the logs from my Synology. I now go over to my pfsense. There a some tutorials out in the www. Unfortunately none of them is working properly or just with old versions. I'm following the tutorial from: [http://pfelk.3ilson.com/](http://pfelk.3ilson.com/) At the moment I recieve the logs in the ELK stack but not parsed correctly. The biggest problem is probably the eventid. ``` filter { if "PFSense" in [tags] { grok { add_tag => ["firewall"] match => ["message", "<(?.*)>(?(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?.*?): (?.*)" ] } mutate { gsub => ["datetime"," "," "] } date { match => ["datetime", "MMM dd HH:mm:ss"] timezone => "America/New_York" } mutate { replace => ["message", "%{msg}"] } mutate { remove_field => ["msg", "datetime"] } } if [prog] =~ /^filterlog$/ { mutate { remove_field => ["msg", "datetime"] } grok { patterns_dir => "/etc/logstash/conf.d/patterns" match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}", "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}" ] } mutate { lowercase => ['proto'] } geoip { add_tag => ["GeoIP"] source => "src_ip" # Optional GeoIP database # Comment out the below if you do not wise to utilize and omit last three steps dealing with (recommended) suffix database => "/etc/logstash/GeoLite2-City.mmdb" } } } ``` I made a picture from the view in Kibana at the moment. I tried with: BASE10NUM but that doesn't work. ![](https://us1.discourse-cdn.com/elastic/original/3X/e/d/ed43590340bb4624a4bc6569abbd2a9ef12bfbf0.jpg) Can someone help me. This is going to be more difficult I think. But I have all the grok patterns from the tutorial and config. Maybe I just have to make a few adjustments. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 28, 2017, 8:06am UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/2 "2017-04-28T08:06:07Z")
> [@hispeed](#): > > match =\> ["message", "\<(?\._)\>(?\(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\s+(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]) (?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:[0-5][0-9])) (?\._?): (?\.\*)" ] This is a bit of a messy grok pattern. Why not use the inbuilt time patterns? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 28, 2017, 8:17am UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/3 "2017-04-28T08:17:16Z")
It looks like you have a kv filter somewhere that creates a field named `<134>Apr 27 20`. Start by fixing that. Next time save your screenshots and PNG instead of JPEG to make them more readable. ---
**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Post date:** [April 28, 2017, 10:47am UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/4 "2017-04-28T10:47:33Z")
warkolm, inbuilt time patterns, what do you mean with that? Can you do an example? @Magnus I have checked it again. I can't see any KV filter in the whole code. Did you look at the website I mentioned? I use the same code as he does. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 28, 2017, 11:01am UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/5 "2017-04-28T11:01:16Z")
> warkolm, inbuilt time patterns, what do you mean with that? He's talking about the stock grok patterns that ship with Logstash, for example these: [https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns) > I have checked it again. I can't see any KV filter in the whole code. What about the rest of your configuration? What's the result of `grep kv /etc/logstash/conf.d`? ---
**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Post date:** [May 2, 2017, 10:30am UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/6 "2017-05-02T10:30:34Z")
> What about the rest of your configuration? What's the result of grep kv /etc/logstash/conf.d? > When i do that, i recieve: No file or folder has been found? I checked if there is any KV filter in the files but I can't find anything. Any other ideas? I will check that today in the evening what warkolm is writing. ---
**Author:** ![hispeed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hispeed/32/16232_2.png) [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Post date:** [May 10, 2017, 6:50pm UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/7 "2017-05-10T18:50:09Z")
I found the problem after hours of searching and trying. If there is someone else who needs with Pfsense 2.3+ and CentOS 7 on ELK Stack 5.2+ you can contact me here or go to my website. I'm going to publish there soon a tutorial for CentOS and ELK. [www.hs-x.ch](http://www.hs-x.ch) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 7, 2017, 6:51pm UTC](https://discuss.elastic.co/t/pfsense-2-3-3-and-elk/83928/8 "2017-06-07T18:51:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.