Dns timeout error

[vaishnavi]

Hi,
I am using logstash to process large files which involves many IPs and I need to convert IP to hostname in logstash using dns filter without any time delay
I am already using DNS filter which convert IP to hostname but for unresolved IPS it take 5 to 15 seconds and displays time out error.
For nearly 1000 IPs it takes a minimum of 1 hour to push data to elasticsearch
I need to push data in faster manner and how can I achieve this?

currently I am using centos 7, logstash 5.2 , elasticsearch 5.0.1

My conf file is

dns {
reverse => "source_ip_dns"
timeout =>0.1
hit_cache_size => 4096
hit_cache_ttl => 900
failed_cache_size => 512
failed_cache_ttl => 900
action => "replace"
}

Any help would be appreciated

[danlobo]

I'm having the same issue.

My setup involves a Redis broker, and when the DNS plugin is enabled the buffer keeps increasing for about an hour. This doesn't happen when the filter is disabled.

Changed some parameters, like hit cache size and failed cache size; also installed a local dns cache (dnsmasq), but no improvements.

[Edit]

GET /_node/stats/pipeline

{
    version: "5.2.2",
    <...>
    pipeline: {
        events: {
            duration_in_millis: 460233,
            in: 11904,
            filtered: 1664,
            out: 1664
        },
        plugins: {
        inputs: [ ],
        filters: [
            <...>
            {
                id: "<...>",
                events: {
                    duration_in_millis: 5820837,
                    in: 1386,
                    out: 1306
                },
                name: "dns"
            },
            <...>
        }]
    }
}