Unable to parse Fortinet Netflow with logstash module

[Blason (R)]

Hi,

I am trying to monitor Fortinet Netflow logs using elasticsearch stack 6.2.2. I am using logstash Netflow module but logs are not completely being parsed and seeing below errors.

Any clue what could be the issue?

[WARN ] 2018-03-07 23:50:25.836 [<udp.0] netflow - Can't (yet) decode flowset id 258 from source id 1, because no template to decode it with has been received. This message will usually go away after 1 minute.
[WARN ] 2018-03-07 23:50:25.836 [<udp.0] netflow - Can't (yet) decode flowset id 258 from source id 1, because no template to decode it with has been received. This message will usually go away after 1 minute.
[WARN ] 2018-03-07 23:50:25.836 [<udp.0] netflow - Can't (yet) decode flowset id 258 from source id 1, because no template to decode it with has been received. This message will usually go away after 1 minute.

[paz]

As it mentions, Logstash has not received the appropriate template for that flowset.

The way Netflow v9 works is, it relies to the device to communicate the template for each different flowset id, basically something like "This flowset number contains x, y, and z fields" so the collector (Logstash in this case) knows how to properly decode the binary information in the payload.

Usually the devices are configured to sent templates for each flowset id each X seconds/minutes. It's usual that you encounter such messages shortly after starting Logstash if no netflow template caching is enabled in the config (until it received the first template for that flowset).
Does this problem persist after a considerable amount of time?

[rcowart (Rob Cowart)]

@paz is spot on. The only thing that I will add is that Fortinet doesn't send netflow templates very often. So with other devices those messages will go away within a couple minutes. But don't be surprised if Fortinet needs 15 minutes or longer.

[Blason (R)]

@rcowart @paz it did stay when I used with Logstash default netflow module while with @rcowart it was spot on and successful.

But @rcowart I was still couldnt configure with Variables may be document is not that clear as I am implementing on CentoS 7.4 and logstash failed to start when I used default conf files but then when I replaced variables with actual stuff like elastic and kibana IP it starts working fine.

Any ways great work @rcowart