Logstash cidr filter throws error if dictionary file is empty

Elastic Stack Logstash
1

This is on the back of an older unresolved issue: Logstash translate filter throws error if dictionary file is empty

I am running into the same problem as mentioned in the above stale issue.

I have a translate filter like so:

 # Check if destination IP address is private.
              cidr {
                id => "elastiflow_postproc_cidr_dst_addr"
                address => [ "%{[flow][dst_addr]}" ]
                # network => [ "0.0.0.0/32", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "fc00::/7", "127.0.0.0/8", "::1/128","169.254.0.0/16", "fe80::/10","224.0.0.0/4", "ff00::/8","255.255.255.255/32" ]
                network_path => "${ELASTIFLOW_DICT_PATH:/usr/share/logstash/elastiflow/dictionaries}/private_ip_addresses.yml"
                refresh_interval => 30
                add_field => { "[flow][dst_autonomous_system]" => "private" }
              }

The file private_ip_addresses.yaml is by default an empty file and will be populated at a later point in time by a different application.

However, when the file is empty, logstash crashes while reading that file: (click to expand error)

Summary 
events, please check your filter configuration and restart Logstash.
org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `collect' for nil:NilClass
	at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_cidr_minus_3_dot_1_dot_2_minus_java.lib.logstash.filters.cidr.filter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-cidr-3.1.2-java/lib/logstash/filters/cidr.rb:144) ~[?:?]
	at usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:143) ~[?:?]
	at usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:162) ~[?:?]
	at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1792) ~[jruby-complete-9.2.7.0.jar:?]
	at usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159) ~[?:?]
	at org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multi_filter(org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:115) ~[logstash-core.jar:?]
	at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:235) ~[?:?]
warning: thread "[elastiflow]>worker0" terminated with exception (report_on_exception is true):
java.lang.IllegalStateException: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `collect' for nil:NilClass
	at org.logstash.execution.WorkerLoop.run(org/logstash/execution/WorkerLoop.java:85)
	at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
	at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:425)
	at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:292)
	at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:235)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)
	at java.lang.Thread.run(java/lang/Thread.java:748)

How exactly do I get it to work?

I want to make it work as below:

  1. if the file is NOT present, fallback on the inline parameters provided (the line that is commented #network)
  2. If the file is present BUT empty, do the same as option 1, fallback to whatever is inline.
  3. If the file is present and has content, rely on the content of the file

Is this doable. How?

Thanks.

2

No, it is not. load_file is careful to handle the case of an empty file, but the filter function is not. It would be trivial to modify load_file to set network_list to an empty array if the file is empty, but the code does not currently do that.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.