One index/index template per module

Hello,

I'd like to enable just a few filebeat modules (let's say nginx, postgresql). And I'd like to have on Elasticsearch one index per module (per month). So... filebeat-nginx-2022-01 and filebeat-postgresql-2022-01. Also, I'd like the index template for these indexes to only include the fields the index will contain -- so only the nginx fields for filebeat-nginx for instance.
I don't want ILM.

I would have actually expected filebeat to include in the index template it generates only the fields from the modules I've enabled, but instead of that, I currently get fields for all the modules that filebeat supports.
I did all my changes in /etc/filebeat/filebeat.yml; I have tried disabling ILM, using %{[event.module]:other} and %{[fields.type]:logs} patterns in my index name, also listing multiple indices in filebeat.yml, with conditions like:

      when.equals:
        event.module: "nginx"

But no matter what I do, I always end up with one index with all the filebeat-supported fields in the index template.

This is my current filebeat.yml file:

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

filebeat.config.inputs:
  enabled: true
  path: inputs.d/*.yml

output.elasticsearch:
  hosts: ["xxxx:9200"]
  # index: "filebeat-%{[fields.type]:logs}-"
  # index: "filebeat-%{[agent.version]}-%{[event.module]:other}-%{+yyyy.MM.dd}"
  indices:
    - index: "filebeat-nginx-%{+yyyy.MM.dd}"
      when.equals:
        event.module: "nginx"
    - index: "filebeat-system-%{+yyyy.MM.dd}"
      when.equals:
        event.module: "system"

  protocol: "https"

setup.template.name: "filebeat-%{[agent.version]}-%{[event.module]:other}"
setup.template.pattern: "filebeat-%{[agent.version]}-%{[event.module]:other}"
setup.ilm.enabled: false
ilm.enabled: false

I tried all the things I could find from the docs and various forums and google results, and now I'm out of ideas. What am I missing?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.