Grok issues in 5.6.3 -> 6.0

Recently, I've made the move to LS 6.0. I was coming from LS 5.6.3 and had a filter working perfectly parsing pfsense log files. I copied over the same file from the 5.6.3 install to use in LS 6.0 install but in 6.0 it's wildly off in how it grabs the field data. Here's how I have the filter starting and it seems to be that this is the point where things get thrown off course.

filter {
if "pfsense" in [tags] {
grok {
match => [ 'message', '.* %{WORD:program}:%{GREEDYDATA:rest}' ]
if [program] == "filterlog" {
# Grab fields up to IP version. The rest will vary depending on IP version.
grok {
match => [ 'rest', '%{INT:rule_number},%{INT:sub_rule_number},,%{INT:tracker_id},%{WORD:interface},%{WORD:reason},%{WORD:action},%{WORD:direction},%{WORD:ip_version},%{GREEDYDATA:re$
mutate {
replace => [ 'message', '%{rest2}' ]

And it continues on from there.

Sample data:
Dec 5 13:41:14 filterlog: 9,16777216,,1000000103,em1,match,block,in,4,0x0,,64,3925,0,DF,6,tcp,52,,,55637,443,0,FA,2928013973,2950294831,4096,,nop;nop;TS

I should get something in ES like:
action: block
direction: in
ip.version: 4
protocol: tcp 1000000103

and so on and so on.

Instead, I get:
action: 2928013973
direction: 2950294831
ip.version: 4096 443

Admittedly, my grok skills are not all that good, so it's entirely possible I fudged something that, while it worked in 5.6.3, fails in 6.0

I did try this out on grok tester at and it parsed correctly at the first gate (program was matched to filterlog, everything else went to rest to be processed further down). So I'm a little puzzled. Any ideas?

Thanks in advance!

Have you tried anchoring the regular expression at the start with a ‘^’?

You might as well use a csv filter for this, but if you insist on grok you might have luck starting your expression with ^ to anchor it to the beginning of the string and make sure %{INT:rule_number} matches "9" and nothing else.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.