Grok Patterns? for OpenVPN

Elastic Stack Logstash
1

Hi,
I Have been trying for many months and no luck on getting this to
work to filter the logs from pfSense and Send them to ELK. I am not
sure where i should be looking at? This is what I have

filter{ if [type] == "OpenVPN_log" { #drop{} grok{ match => { "message" => "%{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR} %{USERNAME}/%{IP:src_ip}:%{BASE10NUM} TLS: Username/Password authentication succeeded for username '%{USERNAME:username}' [CN SET]"} add_tag => "VPN_Success" } }}
But on ELK it showsing something differnt on the RAW

{ "_index": "logstash-2016.06.08", "_type": "syslog", "_id": "Jo75DVZpR-iVZ6yBmFu2dg", "_score": null, "_source": { "message": "181.137.XXX.XXX:52777 [casa] Peer Connection Initiated with [AF_INET]181.137.XXX.XXX:52777", "@version": "1", "@timestamp": "2016-06-08T22:28:53.000Z", "type": "syslog", "host": "192.168.3.254", "tags": [ "PFSense", "firewall" ], "evtid": "29", "prog": "openvpn[50853]" }, "sort": [ 1465424933000, 1465424933000 ] }

and this

{ "_index": "logstash-2016.06.08", "_type": "syslog", "_id": "KkzovBpTRFS53tIxFYUHmA", "_score": null, "_source": { "message": "user 'casa' authenticated", "@version": "1", "@timestamp": "2016-06-08T22:28:53.000Z", "type": "syslog", "host": "192.168.3.254", "tags": [ "PFSense", "firewall" ], "evtid": "37", "prog": "openvpn" }, "sort": [ 1465424933000, 1465424933000 ] }

Clipboarder.2016.06.08.png1308×430 16.8 KB

Thank you

2

Hi,

I am not sure if I understood correctly, but you cannot parse the PfSense logs on ELK.
First, if the information you have provided :

But on ELK it showsing something differnt on the RAW>

is the raw message, then your filter will never work as you apply the filter only for OpenVPN_log type of logs.

Again, I am not sure if I understood correctly.

Secondly, something else worth mentioning is the way that you push your PfSense logs to ELK. You can push them on different formats and based on what you are after, get the best result.
Personally, I use the csv filter for pfsense logs, to parse traffic and snort data, but have never used openvpn on that.

3

Hi,
Thank you for the reply,
Sorry for not being clear,
Well when you say cannot parse the pfSense Logs on to ELK I forgot to mention that Im running logstash with grok patterns. Right now I have working the firewall logs no issue there I did someone got working with OpenVPN. Also been trying with Snort logs to parse also no luck there either. These are my logstash and grok patterns

logstash----http://pastebin.com/5tfVr1eF
grok patterns------http://pastebin.com/2ZQJbAL1

Thank you