Logstash crashes in dns filter

Hi all,

Trying the dns filter in Logstash for the first time, and I'm getting this error in the log:

[2017-07-27T14:54:43,886][ERROR][logstash.pipeline ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {"exception"=>"", "backtrace"=>["java.net.IDN.isRootLabel(java/net/IDN.java:443)", "java.net.IDN.toASCII(java/net/IDN.java:116)", "java.net.IDN.toASCII(java/net/IDN.java:151)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "RUBY.getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:273)", "RUBY.retriable_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:261)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.retriable_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:239)", "org.jruby.ext.timeout.Timeout.timeout(org/jruby/ext/timeout/Timeout.java:115)", "RUBY.retriable_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:238)", "RUBY.retriable_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:260)", "RUBY.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:139)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:122)", "RUBY.filter(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.3/lib/logstash/filters/dns.rb:95)", "RUBY.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145)", "RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161)", "RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:43)", "RUBY.initialize((eval):1916)", "org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)", "RUBY.initialize((eval):1910)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "RUBY.filter_func((eval):608)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:370)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:370)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:224)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:224)", "org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:223)", "LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:223)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:369)", "LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:369)", "RUBY.worker_loop(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:350)", "RUBY.start_workers(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:317)", "java.lang.Thread.run(java/lang/Thread.java:748)"]}

Here's the relevant fragment of config:

filter {
  ...
  dns {
    resolve => [ "source_addr", "dest_addr" ]
    action => "replace"
  }
  ...
}

I don't think this is a duplicate of issue 55223. Does anyone have any ideas how to fix or work around it? I get the same error with logstash 5.4.1 and 5.5.1. The environment is Oracle Java 1.8.0_131 on Debian 8.8

cheers,
Tom

Here's a slightly more readable stacktrace from the console:

Exception in thread "[main]>worker1" java.lang.NullPointerException
at java.net.IDN.isRootLabel(java/net/IDN.java:443)
at java.net.IDN.toASCII(java/net/IDN.java:116)
at java.net.IDN.toASCII(java/net/IDN.java:151)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
at RUBY.getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.
4/lib/logstash/filters/dns.rb:273)
at RUBY.retriable_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filte
r-dns-3.0.4/lib/logstash/filters/dns.rb:261)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
at RUBY.retriable_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-d
ns-3.0.4/lib/logstash/filters/dns.rb:239)
at org.jruby.ext.timeout.Timeout.timeout(org/jruby/ext/timeout/Timeout.java:115)
at RUBY.retriable_request(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-d
ns-3.0.4/lib/logstash/filters/dns.rb:238)
at RUBY.retriable_getaddress(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filte
r-dns-3.0.4/lib/logstash/filters/dns.rb:260)
at RUBY.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.4/l
ib/logstash/filters/dns.rb:139)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at RUBY.resolve(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.4/l
ib/logstash/filters/dns.rb:122)
at RUBY.filter(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-dns-3.0.4/li
b/logstash/filters/dns.rb:95)
at RUBY.do_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145)
at RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161)
at RUBY.multi_filter(/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:43)
at RUBY.initialize((eval):1916)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at RUBY.initialize((eval):1910)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
at RUBY.filter_func((eval):608)
at LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.r
b:383)
at LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.r
b:383)
at org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core
/lib/logstash/util/wrapped_synchronous_queue.rb:238)
at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core
/lib/logstash/util/wrapped_synchronous_queue.rb:238)
at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1342)
at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core
/lib/logstash/util/wrapped_synchronous_queue.rb:237)
at LogStash::Util::WrappedSynchronousQueue::ReadBatch.each(/usr/share/logstash/logstash-core
/lib/logstash/util/wrapped_synchronous_queue.rb:237)
at LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.r
b:382)
at LogStash::Pipeline.filter_batch(/usr/share/logstash/logstash-core/lib/logstash/pipeline.r
b:382)d6beceeb93ade6c3bc18b76a7f0e365dd95f6f52
at RUBY.worker_loop(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:363)
at RUBY.start_workers(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:330)
at java.lang.Thread.run(java/lang/Thread.java:748)

This is happening because one of the fields the filter is trying to resolve is null. We shouldn't crash in that case of course, so I've submitted a patch here: https://github.com/logstash-plugins/logstash-filter-dns/pull/34

You can try to work around this by verifying you don't pass null data to this filter for now.

Ah, awesome, thanks for the quick response!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.