Logstash multiple path.config (copying files is not a valid solution!)

Our logstash configurations are actually splitted into 2 directories:

  • main configuration files.
  • group specific configuration files.

According to docs:

-f, --path.config CONFIG_PATH
Load the Logstash config from a specific file or directory. If a directory is given, all files in that directory will be concatenated in lexicographical order and then parsed as a single config file. Specifying this flag multiple times is not supported. If you specify this flag multiple times, Logstash uses the last occurrence (for example, -f foo -f bar is the same as -f bar).
You can specify wildcards (globs) and any matched files will be loaded in the order described above. For example, you can use the wildcard feature to load specific files by name:
bin/logstash --debug -f '/tmp/{one,two,three}'
With this command, Logstash concatenates three config files, /tmp/one, /tmp/two, and /tmp/three, and parses them into a single config.

but sadly...

bin/logstash --debug -f '/tmp/{firstFile,secondFile}' <---works
bin/logstash --debug -f '/tmp/{firstDir,secondDir}' <--- doesn't work

As the configuration files are stored in host, and logstash is deployed as a container, and considering we are trying to enable automatic-reload of configuration files, does anyone know why this doesn't work or how could we solve this issue?

Issue: https://github.com/elastic/logstash/issues/5097

Regards

We ended not using this feature, as when using multiple paths, logstash concatenates each conf file within each directory, hence having two "configuration sets", instead of one, which was expected/needed.

ie: -f '/tmp/{firstDir,secondDir}' results in first/.conf and second/.conf being created separately.

As we had:
first/10-input.conf
first/30-delete.conf
second/20-input.conf

we expected them to be concat as: first/10-input.conf, second/20-input.conf. first/30-delete.conf

but instead we get: first/10-input.conf, first/30-delete.conf, second/20-input.conf

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.