Recently, I've made the move to LS 6.0. I was coming from LS 5.6.3 and had a filter working perfectly parsing pfsense log files. I copied over the same file from the 5.6.3 install to use in LS 6.0 install but in 6.0 it's wildly off in how it grabs the field data. Here's how I have the filter starting and it seems to be that this is the point where things get thrown off course.
filter {
if "pfsense" in [tags] {
grok {
match => [ 'message', '.* %{WORD:program}:%{GREEDYDATA:rest}' ]
}
if [program] == "filterlog" {
# Grab fields up to IP version. The rest will vary depending on IP version.
grok {
match => [ 'rest', '%{INT:rule_number},%{INT:sub_rule_number},,%{INT:tracker_id},%{WORD:interface},%{WORD:reason},%{WORD:action},%{WORD:direction},%{WORD:ip_version},%{GREEDYDATA:re$
}
mutate {
replace => [ 'message', '%{rest2}' ]
}
And it continues on from there.
Sample data:
Dec 5 13:41:14 pfsense.example.address.com filterlog: 9,16777216,,1000000103,em1,match,block,in,4,0x0,,64,3925,0,DF,6,tcp,52,192.168.0.1,10.1.10.1,55637,443,0,FA,2928013973,2950294831,4096,,nop;nop;TS
I should get something in ES like:
action: block
destination.ip: 10.1.10.1
direction: in
ip.version: 4
protocol: tcp
tracker.id: 1000000103
and so on and so on.
Instead, I get:
action: 2928013973
direction: 2950294831
ip.version: 4096
tracker.id: 443
Admittedly, my grok skills are not all that good, so it's entirely possible I fudged something that, while it worked in 5.6.3, fails in 6.0
I did try this out on grok tester at http://grokconstructor.appspot.com/ and it parsed correctly at the first gate (program was matched to filterlog, everything else went to rest to be processed further down). So I'm a little puzzled. Any ideas?
Thanks in advance!