Hi all,
We're currently using Logstash's NetFlow plugin in order to decode and process all sorts of NetFlow streams (v5, v9, IPFIX) coming from various sources (Checkpoint, Cisco, VMware and others).
We've encountered a problem in NetFlow v9 sent from Cisco routers, where every 5 minutes (or another configurable interval) the router sends a mapping between its interface names and random numbers (standard key-value mapping), and it does the same for VRF names and numbers. In the actual NetFlow packets, we receive a field called "interface_snmp" or "VRF_ID" which is a number.
Is there any way to translate those fields? We've thought about running a ruby code when the mapping messages are being received which will edit a dictionary file in all of the Logstash nodes (then later decode interface numbers using a standard dictionary filter). However, this is rather complex (regardless of performance issues) and our engineers have recommended on simply saving the mappings to a different index and then translate via a python script which will update the documents in the NetFlow index. We have used ElastiFlow in the past and could easily reuse this project again if it offers a solution to this problem...
Any thoughts?