Duplicated Log

If you have a single file output in a configuration file and two copies of the message get written to it then it is very likely you have two configuration files that contain that output.

It is a very common misunderstanding that if you have multiple configuration files they are run independently, but that is not the case unless you are using pipelines.yml. If path.config matches multiple files they are concatenated, events are read from all of the inputs, run through all the filters, and written to all of the events. If two configuration files have the same file output the event will be written twice.

A common way of getting hit by this is to point path.config to a directory, such as /etc/logstash/conf.d/. It will then gather up all the files (e.g. myConfig.conf, myConfig.conf.bak) and combine them.

By the way, if you have two syslog inputs on the same port one of them should be logging an error saying "address already in use".

1 Like