Filebeat: allow prospectors includes from different files

Elastic Stack Beats
1

Filebeat version: 1.2.3 (amd64)
The situation is:
I've got a node on which I start containers with applications on demand. Containers can appear and disappear frequently, and so the log files i need to ship.
What I was trying to do, is to create a path section for appearing application log files dynamically.
Currently config file with prospectors looks like:

filebeat:
  prospectors:
    -
      paths:
        - /mnt/docker/logs/app1/*.log
    - paths:
        - /mnt/docker/logs/app2/*.log

To generate such a file, I need to fetch the list of all the path (all the containers in my situation) every time. I't a bit inconvenient.

It would be cool if filebeat would allow to do path includes.
For example:

/etc/filebeat/filebeat.yml

filebeat: 
config_dir: /etc/filebeat/conf.d

/etc/filebeat/conf.d/app1.yml

  prospectors:
    -
      paths:
        - /mnt/docker/logs/app1/*.log

and /etc/filebeat/conf.d/app2.yml

  prospectors:
    -
      paths:
        - /mnt/docker/logs/app1/*.log

If I do this now, filebeat will use prospectors only from the last config file it read.

UP:
It seems that I was wrong, and mergeConfigs does it properly

2

If I understand you right, config_dir is working as expected. Here are also the docs for it: https://www.elastic.co/guide/en/beats/filebeat/1.2/configuration-filebeat-options.html#_config_dir

3

This topic was automatically closed after 21 days. New replies are no longer allowed.