Logstash stop parsing IIS logs on Windows

[pad (Pierre-Antoine DOUCHET)]

Hi,

I'm using Logstash (2.3.4) on Windows to parse IIS logs (IIS 8.5) on two different server, one on Azure and one on our datacenter.

With the same configuration and the same version of Logstash, the one on our datacenter works correctly but the one on Azure stop reading file once in a while. I have to restart it and the data aren't correctly read.

Is it possible to add some logs on Logstash to see what's happening on the server ?

Thanks for your answers.

[pad (Pierre-Antoine DOUCHET)]

Hi again,

I found how to add logs and I've got my error:
{:timestamp=>"2017-03-31T10:13:17.266000-0500", :message=>"DNS: timeout on resolving address.", :field=>"clientHostname", :value=>"xxx.xxx.xxx.xxx", :level=>:error}

In the conf (the one you can find on the web), I've got this:

## Create a new field for the reverse DNS lookup below
#
add_field => { "clientHostname" => "%{clientIP}" }

## Do a reverse lookup on the client IP to get their hostname.
#
dns {
    ## Now that we've copied the clientIP into a new field we can
    #   simply replace it here using a reverse lookup
    #
    action => "replace"
    reverse => ["clientHostname"]
}

When it can't do the reverse lookup, it seems to stop read the files. Can we tell it to stop the reverse lookup and keep analyse the rest of the line ?

Thanks.

[jsvd (João Duarte)]

Be careful when using the DNS filter since it does a network call for each event, and if the resolve takes time or the dns server doesn't respond, it can clog the pipeline.

This filter has several settings you can tune: it has a timeout options, and it can also have a cache of configurable size to avoid dns requests for repeated ips/hosts.

[pad (Pierre-Antoine DOUCHET)]

Thanks, I will be careful with that option. For now, I deactivate this part to avoid any time out.