Hello, I'm using Logstash 6.3.2 with the latest version of the DNS plugin. I set up Elastiflow to ingest Netflow logs and enabled the DNS lookup option to convert the IP addresses into host names. This appeared to work for about a day but when the DNS plugin encountered a connection refused error on the DNS server IP address it was using (actually a loadbalanced address by a Citrix Netscaler), it completely crashed the Logstash instance.
Is this the expected behaviour?
The partial error log is below:
[2018-12-21T22:19:43,432][WARN ][logstash.filters.dns ] DNS: timeout on resolving address. {:field=>"[flow][dst_hostname]", :value=>"104.25.15.8"}
[2018-12-21T22:19:43,806][WARN ][logstash.filters.dns ] DNS: timeout on resolving address. {:field=>"[flow][dst_hostname]", :value=>"104.25.15.8"}
[2018-12-21T22:19:43,952][WARN ][logstash.filters.dns ] DNS: timeout on resolving address. {:field=>"[flow][dst_hostname]", :value=>"104.25.15.8"}
[2018-12-21T22:19:47,005][WARN ][logstash.filters.dns ] DNS: timeout on resolving address. {:field=>"[flow][dst_hostname]", :value=>"104.16.92.60"}
[2018-12-21T22:19:59,206][ERROR][logstash.pipeline ] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {:pipeline_id=>"elastiflow", "exception"=>"Connection refused - connect(2) for \"192.168.56.3\" port 53", "backtrace"=>["org/jruby/ext/socket/RubyTCPSocket.java:134:in `initialize'", "org/jruby/RubyIO.java:875:in `new'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:872:in `initialize'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:574:in `make_tcp_requester'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:542:in `block in fetch_resource'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:1108:in `block in resolv'", "org/jruby/RubyArray.java:1734:in `each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:1106:in `block in resolv'", "org/jruby/RubyArray.java:1734:in `each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:1105:in `block in resolv'", "org/jruby/RubyArray.java:1734:in `each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:1103:in `resolv'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:527:in `fetch_resource'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:517:in `each_resource'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:471:in `each_name'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:154:in `block in each_name'", "org/jruby/RubyArray.java:1734:in `each'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:153:in `each_name'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:135:in `getname'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:294:in `getname'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:281:in `block in retriable_getname'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:266:in `block in retriable_request'", "org/jruby/ext/timeout/Timeout.java:117:in `timeout'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:265:in `retriable_request'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:280:in `retriable_getname'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:220:in `block in reverse'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:192:in `reverse'", "/usr/local/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-dns-3.0.11/lib/logstash/filters/dns.rb:102:in `filter'", "/usr/local/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/local/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `block in multi_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/local/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'", "/usr/local/logstash/logstash-core/lib/logstash/filter_delegator.rb:44:in `multi_filter'", "(eval):519799:in `block in initialize'", "org/jruby/RubyArray.java:1734:in `each'", "(eval):519796:in `block in initialize'", "(eval):519814:in `block in initialize'", "org/jruby/RubyArray.java:1734:in `each'", "(eval):519810:in `block in initialize'", "(eval):16652:in `block in filter_func'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:443:in `filter_batch'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:422:in `worker_loop'", "/usr/local/logstash/logstash-core/lib/logstash/pipeline.rb:384:in `block in start_workers'"], :thread=>"#<Thread:0x7ca25bf2 sleep>"}
[2018-12-21T22:19:59,691][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<Errno::ECONNREFUSED: Connection refused - connect(2) for "192.168.56.3" port 53>, :backtrace=>["org/jruby/ext/socket/RubyTCPSocket.java:134:in `initialize'", "org/jruby/RubyIO.java:875:in `new'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:872:in `initialize'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/resolv.rb:574:in `make_tcp_requester'", "uri:classloader:/META-