Enable 2 different filebeat modules & send to different index name

Elastic Stack Beats
1

Hello,

I am trying to use a single filebeat, enabled with system & apache module to send to elasticsearch. I would want to have a system index and a apache index. How can i do this?
ILM is enabled for me and for a single module, i set my index via this setting:

setup.ilm.enabled: auto
setup.ilm.rollover_alias: "filebeat-linuxclient"
setup.ilm.pattern: "{now/d}-000001"
setup.ilm.overwrite: false

now i would like to have filebeat-linuxclient-system and filebeat-linuxclient-apache as the alias.

This is my current filebeat.yml file settings for enabling system module:

filebeat.inputs:

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "kibana:5601"

output.elasticsearch:
  hosts: ["esnode1:9200", "esnode2:9200", "esnode3:9200"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

logging.metrics.period: 60

setup.ilm.enabled: auto
setup.ilm.rollover_alias: "filebeat-linuxclient"
setup.ilm.pattern: "{now/d}-000001"
setup.ilm.overwrite: false

How can i do this?

2

Hi,

Does my comment in this thread help with what you're trying to achieve?

3

Should I comment out the ILM settings?

I guess I should write these settings in filebeat.yml and

output.elasticsearch:
  index: "default-%{+yyyy.MM.dd}"
  indices:
    - index: "filebeat-system-%{+yyyy.MM.dd}"
      when.equals:
        application: "system"
    - index: "filebeat-apache-%{+yyyy.MM.dd}"
      when.equals:
        application: "apache"

Do I need to tag application: system and application: apache in their modules config file?

4

I'm not familiar with ILM so I can't say what you'd want to do with that.

application was just an example of a potential field name you could use to differentiate between the two log sources.

I think given you're using two different modules you can probably use the event.module field or something similar to configure different index outputs.

1 Like
5

Alright i will give it a try! Thanks

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.