DNS filter not working

[nigdav007 (NigelD)]

Hay all

I am an tad stuck with the DNS filter.

My issue all my syslogs hosts are coming up with the ip address and not the hostname.

I like to use the DNS filter to get the hostname, but for some reason its not working.

if i use this

filter {
mutate {
add_field => { "hostname" => "%{host}" }
}
dns {
action => "replace"
reverse => [ "hostname" ]
add_tag => [ "dns_lookup" ]
}

}

I get logstash adding the host name field but nothing else

If i do this nothing happens at all

filter {
dns {
add_field => { "hostname" => "%{host}" }
action => "replace"
reverse => [ "hostname" ]
add_tag => [ "dns_lookup" ]
}

}

Its like anything inside the DNS filter is being bypassed?

the version of logstash i am using is 5.2.0

Thanks in advance

[rcowart (Rob Cowart)]

Although the docs would lead one to believe that the name server used by the OS will be used by default by the DNS filter, the only way I have been able to make it work is by specifying a name server in the DNS filter config. The following works for me (logstash 5.0+)...

if [conn][dst_addr] {
    mutate {
        add_field => { "[conn][dst_hostname]" => "%{[conn][dst_addr]}"}
    }
    dns {
        reverse => [ "[conn][dst_hostname]" ]
        action => "replace"
            
        # CUSTOMIZE THE FOLLOWING VALUES AS REQUIRED BY YOUR ENVIRONMENT!
        nameserver => [ "192.168.255.1" ]
        hit_cache_size => 4096
        hit_cache_ttl => 900
        failed_cache_size => 512
        failed_cache_ttl => 900
    }
}

[nigdav007 (NigelD)]

Hay Rcowart

thank for this i just tried and with no luck i changed your filter you gave me to this

filter {

mutate {
    add_field => { "hostname" => "%{host}"}
}
dns {
    reverse => [ "hostname" ]
    action => "replace"

    # CUSTOMIZE THE FOLLOWING VALUES AS REQUIRED BY YOUR ENVIRONMENT!
    nameserver => [ "XX.X.X.X" ]
    hit_cache_size => 4096
    hit_cache_ttl => 900
    failed_cache_size => 512
    failed_cache_ttl => 900
}

}

Can you see an issue with this config?

i also have and extra filter in the config, to just adds an new field what works

[rcowart (Rob Cowart)]

That looks good. In fact I have been using exactly that for months without issue.

It is possible that the DNS lookup is taking too long and timing out. Run this command...

curl -w '\nlookup time:\t%{time_namelookup}\n' -o /dev/null -s http://www.google.com

You will get an output like this...

lookup time:	0.066

This means the name lookup portion of fetching www.google.com took 0.066 seconds. The default timeout for the logstash DNS filter is 0.5 seconds. If your lookups are slower than that it will be as if the filter didn't work. If your DNS lookups are slow try changing the timeout option.

NOTE: If you are suffering from slow lookups you will need to figure out how speed things up, or reconsider using the DNS filter. Slow name lookups will dramatically slow overall event throughput.

Rob

[astickler]

I have found that add_field does not work as expected inside the dns filter. If the target field is referenced in the reverse/resolve option, then it will never appear in the event; if the target field is not referenced by resolve/reverse, it will appear in the event.

I think this partly explains NigelD's original post.
Andrew