Hello to everyone!
Not long ago I started to use Logstash to collect syslog from various devices in our infrastructure.
To solve this problem, I created some amount of pipelines (now about 40) that help me route syslog to different files.
Below you can see the example of pipeline configuration:
input {
udp {
port => 1140
}
}
filter {
dns {
action => "replace"
hit_cache_size => 500000
max_retries => 10
reverse => [ "[host]" ]
}
}
output {
file {
codec => line { format => "%{message}" }
path => "E:/logstash/ans/ans_file_syslog/%{[host]}/%{+YYYY-MM-dd-HH}.log"
}
}
Due to this configuration, I expected to see two behaviors:
- If everything is OK with DNS, Logstash creates a folder with a DNS name of a particular network device
- If something is wrong with the DNS lookup, I will see a folder with an IP address of this network device
To be sure that everything OK, I create a report that tells me about the second unwanted behavior. And the problem is that I observe this behavior every day, with different hosts and without any time correlation.
And the weirdest thing for me is that we have about ~30kk log messages per day, and only a couple of messages from different hosts correspond to described behavior.
So, is something wrong with my configuration?
P.S:
Before we moved to Logstash, I didn't observe such behavior.