The initial symptom I was seeing was logstash throwing NotImplementedError
on receipt of the template packet from a Juniper SRX110 on 12.1X44-D45.2 (not sure if version matters, but this version seems to behave this way 100% consistently):
NotImplementedError: NotImplementedError
sensible_default at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/base_primitive.rb:239
_value at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/base_primitive.rb:145
do_num_bytes at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/base_primitive.rb:135
sum_num_bytes_below_index at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/struct.rb:257
each at org/jruby/RubyRange.java:479
inject at org/jruby/RubyEnumerable.java:852
sum_num_bytes_below_index at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/struct.rb:254
sum_num_bytes_for_all_fields at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/struct.rb:250
do_num_bytes at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/struct.rb:148
num_bytes at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/base.rb:168
decode_netflow9 at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-netflow-3.1.2/lib/logstash/codecs/netflow.rb:282
decode at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-netflow-3.1.2/lib/logstash/codecs/netflow.rb:157
each at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/array.rb:208
each at org/jruby/RubyArray.java:1613
each at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bindata-2.3.4/lib/bindata/array.rb:208
decode at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-netflow-3.1.2/lib/logstash/codecs/netflow.rb:153
inputworker at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-3.0.3/lib/logstash/inputs/udp.rb:102
udp_listener at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-udp-3.0.3/lib/logstash/inputs/udp.rb:73
After adding some instrumentation and capturing the NetFlow packets, I see that the crash appears to occur while parsing an 'options' flowset which appears immediately after its corresponding template in the same packet. I don't really understand BinData's behaviour in calculating num_bytes
, which is where this is crashing, but it doesn't like what I'm sending it for some reason. I've looked over the decoded packet and code, but my understanding of Ruby and particularly BinData hasn't really given me any good idea why this is failing. I think it may have something to do with the scope field having length 0, and therefore having a nil value in the BinData struct, but I'm not sure how to confirm this or to address it.
I've pastebinned the tshark decoded packet here (too long for Discourse posts here, apparently).