Hey, I'm a pretty new to the ELK stack, and I am trying to configure it to take in and make use of netflow records. After some fiddling around, I've got the records making it through the stack, but not all of the fields are staying intact. I'll resort to examples-
This is the netflow record capture by tshark:
Flow 5
Wireless LAN SSId: nolan
Monitoring Interval Start MilliSeconds: Jun 14, 2016 10:19:30.000000000 PDT
SrcAddr: 20.1.1.1
DstAddr: 255.255.255.255
ApplicationID: NBAR Application ID: 13:13 (type:id)
SrcPort: 67
DstPort: 68
Direction: Egress (1)
Protocol: UDP (17)
Wtp Mac Address: f4:4e:05:98:11:d0
Sta Mac Address: 2c:33:7a:5b:46:bb
Metering Process Id: 2467706807
Octets: 328
Packets: 1
And here is logstashes output: {
"@timestamp" => "2016-06-14T17:21:02.000Z",
"netflow" => {
"version" => 9,
"flow_seq_num" => 12324,
"flowset_id" => 256,
"wlanSSID" => "nolan",
"monitoring_interval_start_milli_secs" => "1465924770000",
"ipv4_src_addr" => "20.1.1.1",
"ipv4_dst_addr" => "255.255.255.255",
"application_id" => 218103821,
"l4_src_port" => 67,
"l4_dst_port" => 68,
"direction" => 1,
"protocol" => 17,
"wtpMacAddress" => "\xF4N\x05\x98\x11\xD0",
"staMacAddress" => "48599607756475",
"metering_process_id" => 2467706807,
"in_bytes" => 328,
"in_pkts" => 1
},
"@version" => "1",
"host" => "20.1.1.1"
}
Certain fields that matter to me, namely application id, and the mac addresses, are not making it through correctly. Here is what I have in my yaml file for these fields:
95:
- 4
- :application_id
365: - 6
- staMacAddress
367: - :string
- wtpMacAddress
In my config file I mutate staMacAddress to a string, just cause I was exploring different options. Does anyone have experience or any ideas with how to make these fields come through cleanly, aka retain the correct type and look like my tshark captured records? I appreciate any feedback greatly.