Logstash receives the attached packets (template and flow packets) but it seems to drop the flow packet with the error
[WARN ][logstash.codecs.netflow ] Can't (yet) decode flowset id 2048 from observation domain id 1010, because no template to decode it with has been received. This message will usually go away after 1 minute.
Subsequent flow packets seem to parsed and it is showing up in kibana UI.
I need help in figuring out why this first flow packet alone is dropped even though template packet is coming before the flow packet.
I'm using these versions
I have a netflow codec configuration for UDP messages similar to this
** versions => [5,9,10]**
** target => "ipfix"**
** cache_save_path => "/usr/..."**
** ipfix_definitions => "...path to ipfix.yml file"**
Always, the first flow packet for a new domain ID is dropped by logstash.
This is likely due to the fact that Logstash is multithreaded, and will start multiple input and pipeline.workers. While the template record is being processed by one worker, the flow record is grabbed by another. Since the template isn't processed yet, and available to decode the flow, you get an error.
You could try setting both UDP input and pipeline workers to 1. However I wouldn't suggest it for production. If you have even a moderate volume of flow data, you will need all of the workers (and corresponding cores) as possible.
Currently, Is there any option to preload the template information in logstash configuration ( as I have only 2 different templates for now) or
will there be any future implementation of buffering flows until template arrives (as specified in logstash codec plugin
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.