Template length doesn't fit cleanly into flowset

[tono (Jose Antonio Castillo)]

Hello,

I upgraded to the latest version of logstash (5.4.1) in order to fix an "out of range" error that I was getting when processing netflow packets.

The "out of range" error went away after upgrading the previous logstash versoin (5.3.3) but now I am getting some errors in the logs and I don't see any new data coming in when I look for a firewall that I am using to send logs for testing.

I found other tickets about similar issues to this one but those are for older versions of ELK.

The firewall is an ASAv10 running in AWS
Current logstash version = 5.4.1
Current elasticsearch version = 541
Current kibana version = 5.4.1
OS = ubuntu 16.04

This is just a piece of the log that is showing me the errors:

a piece of the log that is showing the errors:

[2017-06-02T20:26:00,205][WARN ][logstash.codecs.netflow ] No matching template for flow id 256 [2017-06-02T20:26:00,205][WARN ][logstash.codecs.netflow ] No matching template for flow id 263 [2017-06-02T20:26:00,205][WARN ][logstash.codecs.netflow ] No matching template for flow id 256 [2017-06-02T20:26:00,205][WARN ][logstash.codecs.netflow ] No matching template for flow id 263 [2017-06-02T20:26:00,205][WARN ][logstash.codecs.netflow ] No matching template for flow id 260 [2017-06-02T20:26:00,205][WARN ][logstash.codecs.netflow ] No matching template for flow id 256 [2017-06-02T20:26:00,205][WARN ][logstash.codecs.netflow ] No matching template for flow id 263 [2017-06-02T20:26:00,207][WARN ][logstash.codecs.netflow ] No matching template for flow id 263 [2017-06-02T20:26:00,258][WARN ][logstash.codecs.netflow ] No matching template for flow id 256 [2017-06-02T20:26:00,258][WARN ][logstash.codecs.netflow ] No matching template for flow id 263 [2017-06-02T20:26:00,258][WARN ][logstash.codecs.netflow ] No matching template for flow id 256 [2017-06-02T20:26:00,258][WARN ][logstash.codecs.netflow ] No matching template for flow id 263 [2017-06-02T20:26:00,258][WARN ][logstash.codecs.netflow ] No matching template for flow id 256 [2017-06-02T20:26:00,258][WARN ][logstash.codecs.netflow ] No matching template for flow id 263 [2017-06-02T20:26:07,020][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>88} [2017-06-02T20:26:07,345][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:07,640][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:07,991][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:08,226][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>260} [2017-06-02T20:26:18,641][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:18,907][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:19,072][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:19,270][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:19,598][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>172} [2017-06-02T20:26:22,114][WARN ][logstash.codecs.netflow ] Template length doesn't fit cleanly into flowset {:template_id=>263, :template_length=>62, :record_length=>88}

This is my input filter:

input {
udp {
port => 5555
type => netflow
codec => netflow {
netflow_definitions => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-netflow-3.4.0/lib/logstash/codecs/netflow/netflow.yaml"
versions => [9]
}
}
}


output {
elasticsearch {
hosts => ["xx.xx.xx.xx","xx.xx.xx.xx"]
index => "netflow-%{+YYYY.MM.dd}"
user => "some_user"
password => "some_pass"
}
}


(I apologize, I couldn't format the logs and filter for better reading correctly)

This is affecting the vilsualizations I created in kibaba as they are not showing data for 1 of the 2 firewalls that are currently sending netflow data to the logstash server.

Any ideas?

Thank you.