Multiple configs in /etc/filebeat/*.yml

[jvdm (Jacques van der Merwe)]

Greetings all,

I'm trying to use multiple config files located in /etc/filebeat. The default config is stored as filebeat.yml, with additional configs stored as $service.yml. I've verified that all the configs are working and functional by using the "-c config.yml -e -d "*"" switches against each .yml. When starting filebeat, only one prospector and accompanying paths from filebeat.yml are active. I'm using filebeat version 5.0.0-rc1.

I'm trying to use the same approach with packetbeat and metricbeat, whereby system defaults are located in *beat.yml, and additional services are defined in $service.yml via configuration management (ansible).

Any help/suggestions would be most welcome!

-Jacques

[ruflin]

What is your exact command that you use to start filebeat etc.?

[jvdm (Jacques van der Merwe)]

Hi @ruflin,

"/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat"

I've tried pointing -path.config to another location other than /etc/filebeat as well.

It's launched via systemctl, and I've tried manually on the command line as well...

[ruflin]

I think you are mixing here two things:

• You can use the -c option multiple times to chain multiple config files: https://www.elastic.co/guide/en/beats/libbeat/5.0/config-file-format-cli.html This works for all beats
• The option you are probably looking for in filebeat is config_dir: https://www.elastic.co/guide/en/beats/filebeat/5.0/configuration-global-options.html#_config_dir This currently does not exist for the other beats
• Defining path.config only tells the beat where to look for the beatname.yml file.

Let me know if the above helps.

[jvdm (Jacques van der Merwe)]

This does help indeed, thanks! The docs that I referenced previously did not mention the config_dir option, and eluded to the -path.config switch instead.

Regarding the other beats, how does one take a modular approach with the configs?

[ruflin]

For the other beats currently the best approach is using the chaining of config files with -c. Or as you alreasy use ansible it should be possible to use ansible to generate one big config file.

What docs were you referencing to initially where the config_dir was not mentioned?