Parsing USG Pro Firewall logs using GROK

[jaysbeekay (Jonathan)]

I have managed to stand up an ELK stack on an Ubuntu host to bring in my NGINX logs, which is all working well.

I have now tried to expand this to bring in firewall logs from my USG Pro Firewall and am running into some issues when trying to formulate a GROK formula!

Example log:

Feb 12 10:08:12 USG-Pro-4 kernel: [WAN_OUT-2000-D]IN=eth0.20 OUT=eth2 MAC=fc:ec:da:48:75:63:d8:0d:17:f0:2f:cd:08:00:45:00:00:2c SRC=192.168.20.69 DST=13.200.17.13 LEN=44 TOS=0x00 PREC=0x00 TTL=63 ID=58512 PROTO=TCP SPT=59296 DPT=443 WINDOW=8192 RES=0x00 SYN URGP=0

Using GROK Debug, I have managed to construct the following grok formula which parses everything from [WAN_OUT-2000] onwards correctly:

\[%{WORD:interface}-%{WORD:ruleindex}-%{WORD:action}\]IN=%{NOTSPACE:IN}%{SPACE}OUT=%{NOTSPACE:OUT}%{SPACE}MAC=%{NOTSPACE:MAC}%{SPACE}SRC=%{NOTSPACE:SRCIP}%{SPACE}DST=%{NOTSPACE:DSTIP}%{SPACE}LEN=%{WORD:LEN}%{SPACE}TOS=%{WORD:TOS}%{SPACE}PREC=%{WORD:PREC}%{SPACE}TTL=%{WORD:TTL}%{SPACE}ID=%{WORD:ID}%{SPACE}(%{NOTSPACE:PROTO}| DF PROTO)=%{WORD:DF}%{SPACE}SPT=%{WORD:SPT}%{SPACE}DPT=%{WORD:DPT}

Any ideas on how to parse the date, hostname and source would be greatly appreciated!

[Badger]

Do not start with grok. Think about using dissect and kv. Here is an example.

grok is massively overused because it is extremely powerful and one of the earliest options. That does not make it a good solution.

[jaysbeekay (Jonathan)]

Thanks Badger, I'm sooo close with Grok was hoping it was something simple that I needed to add.

If I can't figure it out by the end of this week will have to investigate dissect.