Logstash hangs using dns filter

Elastic Stack Logstash
1

Hi there, -Hope this is the right place to put my problem-

I am using Logstash 6.0.0 to parse logs from Filebeat 6.0.0 instances. I tried to change some IP addresses with server names using the DNS filter but after a while processing (less than a day), Logstash stops writing logs and sending messages to ElasticSearch (also 6.0.0). But even without -apparently- doing anything, Logstash is still running and need a kill -9 to stop...

I have two logstash instances running on RedHat7 servers with Xms and Xmx configured to 8g.
I assume that my problem is related to the DNS filter because when I comment DNS configuration lines, everything works fine.

Here are the parts of my logstash configuration using DNS filter :

  grok {
    match => { "message" => [
      "^%{GREEDYDATA:clientip}, %{IPORHOST:ident} - - \[%{HTTPDATE:log_date}\] %{NUMBER:response_time:int} %{QS:request} %{NUMBER:response:int} (?:%{NUMBER:bytes:int}|-) (%{QS:referrer}|-) %{QS:agent}$",
      "^%{GREEDYDATA:clientip} - - \[%{HTTPDATE:log_date}\] %{NUMBER:response_time:int} %{QS:request} %{NUMBER:response:int} (?:%{NUMBER:bytes:int}|-) (%{QS:referrer}|-) %{QS:agent}$"
      ] }
    tag_on_failure => ["_apacheparsingfailure"]
  }
#  dns {
#    reverse => [ "clientip" ]
#    action => "replace"
#  }

And in the exact same way for other logs :

#    dns {
#      reverse => [ "hostaddr" ]
#      action => [ "replace" ]
#    }

We have a old version (Logstash 1.5.2) of the stack running (Which I did not work on) that does not have this problem.
I was wondering if it could be a DNS server problem because I receive avout 10 000 messages/5min that pass in one of the two filters here, but I do not know if it could be too much or not. (The working old version had about 4 000 messages/5min)
Those part of my configuration are copy-pasting of what was used with Logstash 1.5.2 (before the decision to restart the whole project with new versions of tools and new servers) so I might have missed a breaking change or something.

Before asking here I made some searches and found some similar problems, not necessarily related to dns, but couldn't resolve mine...
Here are some of those links :
https://discuss.elastic.co/t/logstash-stops-processing-files-after-a-while/944/3
https://github.com/logstash-plugins/logstash-input-lumberjack/issues/9
https://github.com/elastic/logstash/issues/3914

I think that if there really was a bug somewhere someone would have hit it before me, so I hope that someone here knows this issue or understand it and could give me a workaround

Any help would be VERY appreciated

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.