Although the docs would lead one to believe that the name server used by the OS will be used by default by the DNS filter, the only way I have been able to make it work is by specifying a name server in the DNS filter config. The following works for me (logstash 5.0+)...
if [conn][dst_addr] {
mutate {
add_field => { "[conn][dst_hostname]" => "%{[conn][dst_addr]}"}
}
dns {
reverse => [ "[conn][dst_hostname]" ]
action => "replace"
# CUSTOMIZE THE FOLLOWING VALUES AS REQUIRED BY YOUR ENVIRONMENT!
nameserver => [ "192.168.255.1" ]
hit_cache_size => 4096
hit_cache_ttl => 900
failed_cache_size => 512
failed_cache_ttl => 900
}
}
This means the name lookup portion of fetching www.google.com took 0.066 seconds. The default timeout for the logstash DNS filter is 0.5 seconds. If your lookups are slower than that it will be as if the filter didn't work. If your DNS lookups are slow try changing the timeout option.
NOTE: If you are suffering from slow lookups you will need to figure out how speed things up, or reconsider using the DNS filter. Slow name lookups will dramatically slow overall event throughput.
I have found that add_field does not work as expected inside the dns filter. If the target field is referenced in the reverse/resolve option, then it will never appear in the event; if the target field is not referenced by resolve/reverse, it will appear in the event.
I think this partly explains NigelD's original post.
Andrew
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.