This is on the back of an older unresolved issue: Logstash translate filter throws error if dictionary file is empty
I am running into the same problem as mentioned in the above stale issue.
I have a translate filter like so:
# Check if destination IP address is private.
cidr {
id => "elastiflow_postproc_cidr_dst_addr"
address => [ "%{[flow][dst_addr]}" ]
# network => [ "0.0.0.0/32", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "fc00::/7", "127.0.0.0/8", "::1/128","169.254.0.0/16", "fe80::/10","224.0.0.0/4", "ff00::/8","255.255.255.255/32" ]
network_path => "${ELASTIFLOW_DICT_PATH:/usr/share/logstash/elastiflow/dictionaries}/private_ip_addresses.yml"
refresh_interval => 30
add_field => { "[flow][dst_autonomous_system]" => "private" }
}
The file private_ip_addresses.yaml is by default an empty file and will be populated at a later point in time by a different application.
However, when the file is empty, logstash crashes while reading that file: (click to expand error)
Summary
events, please check your filter configuration and restart Logstash.
org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `collect' for nil:NilClass
at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_cidr_minus_3_dot_1_dot_2_minus_java.lib.logstash.filters.cidr.filter(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-cidr-3.1.2-java/lib/logstash/filters/cidr.rb:144) ~[?:?]
at usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:143) ~[?:?]
at usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:162) ~[?:?]
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1792) ~[jruby-complete-9.2.7.0.jar:?]
at usr.share.logstash.logstash_minus_core.lib.logstash.filters.base.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159) ~[?:?]
at org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multi_filter(org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:115) ~[logstash-core.jar:?]
at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:235) ~[?:?]
warning: thread "[elastiflow]>worker0" terminated with exception (report_on_exception is true):
java.lang.IllegalStateException: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method `collect' for nil:NilClass
at org.logstash.execution.WorkerLoop.run(org/logstash/execution/WorkerLoop.java:85)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:425)
at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:292)
at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:235)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)
at java.lang.Thread.run(java/lang/Thread.java:748)
How exactly do I get it to work?
I want to make it work as below:
- if the file is NOT present, fallback on the inline parameters provided (the line that is commented #network)
- If the file is present BUT empty, do the same as option 1, fallback to whatever is inline.
- If the file is present and has content, rely on the content of the file
Is this doable. How?
Thanks.